Jul 01 17:13:32.619665 ip-10-0-139-93 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 01 17:13:32.619677 ip-10-0-139-93 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 01 17:13:32.619687 ip-10-0-139-93 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 01 17:13:32.620017 ip-10-0-139-93 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 01 17:13:42.841325 ip-10-0-139-93 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 01 17:13:42.841343 ip-10-0-139-93 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 968702a3b0c94108b5106694437af577 --
Jul 01 17:15:36.869515 ip-10-0-139-93 systemd[1]: Starting Kubernetes Kubelet...
Jul 01 17:15:37.399301 ip-10-0-139-93 systemd[1]: Started Kubernetes Kubelet.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.319162    2585 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322824    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322835    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322840    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322843    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322847    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322850    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322853    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.410584 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322856    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322859    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322862    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322865    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322868    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322871    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322874    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322877    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322880    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322883    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322885    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322888    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322891    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322894    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322896    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322899    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322902    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322904    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322908    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322918    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.411825 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322921    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322923    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322925    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322928    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322931    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322933    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322936    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322938    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322940    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322944    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322946    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322949    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322951    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322954    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322958    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322962    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322965    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322967    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322970    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.412762 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322974    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322977    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322979    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322982    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322985    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322987    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322992    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322995    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.322998    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323001    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323004    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323007    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323010    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323012    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323015    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323018    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323020    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323023    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323025    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.413570 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323028    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323030    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323033    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323037    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323040    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323043    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323045    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323050    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323055    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323058    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323061    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323064    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323066    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323069    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323072    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323074    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323077    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323080    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323083    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323086    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.414613 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323089    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323091    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323094    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323096    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323099    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323101    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323104    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323106    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323109    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323111    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323114    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323116    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323119    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323121    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323859    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323865    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323868    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323873    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323878    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.416012 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323882    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323886    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323889    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323891    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323894    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323896    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323900    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323903    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323906    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323909    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323912    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323915    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323917    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323920    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323923    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323925    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323928    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323931    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323933    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323935    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.416994 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323938    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323940    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323943    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323945    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323948    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323951    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323953    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323956    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323958    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323960    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323963    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323966    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323969    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323972    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323974    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323977    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323979    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323982    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323985    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.418235 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323988    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323990    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323994    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323997    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.323999    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324002    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324004    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324007    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324010    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324012    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324015    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324017    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324021    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324025    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324028    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324031    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324033    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324036    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324039    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324042    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.419468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324046    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324048    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324051    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324054    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324056    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324059    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324062    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324064    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324067    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324070    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324072    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324075    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324077    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324080    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324083    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324085    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324088    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324091    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324093    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324096    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.421006 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324100    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324103    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324106    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324109    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324112    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324115    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324120    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324123    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324126    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324129    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324132    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324134    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324137    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324140    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324142    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324598    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324603    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324606    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324609    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.422161 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324612    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324615    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324617    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324620    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324622    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324625    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324628    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324631    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324634    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324637    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324639    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324642    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324645    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324647    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324650    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324653    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324655    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324658    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324660    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324665    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.426884 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324667    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324670    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324672    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324675    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324677    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324680    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324682    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324684    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324687    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324689    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324692    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324694    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324697    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324701    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324704    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324706    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324710    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324714    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324717    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.427930 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324720    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324723    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324725    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324728    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324730    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324734    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324736    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324739    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324741    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324744    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324746    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324749    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324752    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324755    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324757    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324760    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324762    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324765    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324767    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324769    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.582119 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324772    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324774    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324777    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324779    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324781    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324784    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324788    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324790    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324793    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324795    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324798    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324800    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324803    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324806    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324808    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324811    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324813    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324815    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324818    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324820    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.600734 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324823    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324825    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324828    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324831    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324836    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324840    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324842    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324845    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324848    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324850    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324853    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324855    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324859    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324862    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324864    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.324867    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324938    2585 flags.go:64] FLAG: --address="0.0.0.0"
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324953    2585 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324964    2585 flags.go:64] FLAG: --anonymous-auth="true"
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324971    2585 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 01 17:15:37.601456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324979    2585 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324986    2585 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.324997    2585 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325003    2585 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325008    2585 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325013    2585 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325019    2585 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325023    2585 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325026    2585 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325029    2585 flags.go:64] FLAG: --cgroup-root=""
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325033    2585 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325036    2585 flags.go:64] FLAG: --client-ca-file=""
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325039    2585 flags.go:64] FLAG: --cloud-provider="external"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325042    2585 flags.go:64] FLAG: --cluster-dns="[]"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325047    2585 flags.go:64] FLAG: --cluster-domain=""
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325049    2585 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325054    2585 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325057    2585 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325060    2585 flags.go:64] FLAG: --container-log-max-files="5"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325064    2585 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325067    2585 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325070    2585 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325073    2585 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325076    2585 flags.go:64] FLAG: --contention-profiling="false"
Jul 01 17:15:37.602570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325079    2585 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325082    2585 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325085    2585 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325088    2585 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325105    2585 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325108    2585 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325113    2585 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325116    2585 flags.go:64] FLAG: --enable-load-reader="false"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325119    2585 flags.go:64] FLAG: --enable-server="true"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325122    2585 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325126    2585 flags.go:64] FLAG: --event-burst="100"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325129    2585 flags.go:64] FLAG: --event-qps="50"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325132    2585 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325135    2585 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325138    2585 flags.go:64] FLAG: --eviction-hard=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325142    2585 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325145    2585 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325148    2585 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325151    2585 flags.go:64] FLAG: --eviction-soft=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325154    2585 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325157    2585 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325159    2585 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325162    2585 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325165    2585 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325181    2585 flags.go:64] FLAG: --fail-swap-on="true"
Jul 01 17:15:37.603300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325184    2585 flags.go:64] FLAG: --feature-gates=""
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325188    2585 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325191    2585 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325194    2585 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325198    2585 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325201    2585 flags.go:64] FLAG: --healthz-port="10248"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325204    2585 flags.go:64] FLAG: --help="false"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325207    2585 flags.go:64] FLAG: --hostname-override="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325210    2585 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325213    2585 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325216    2585 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325219    2585 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325222    2585 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325227    2585 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325230    2585 flags.go:64] FLAG: --image-service-endpoint=""
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325233    2585 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325235    2585 flags.go:64] FLAG: --kube-api-burst="100"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325238    2585 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325241    2585 flags.go:64] FLAG: --kube-api-qps="50"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325244    2585 flags.go:64] FLAG: --kube-reserved=""
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325247    2585 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325250    2585 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325253    2585 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325257    2585 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 01 17:15:37.604055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325260    2585 flags.go:64] FLAG: --lock-file=""
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325262    2585 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325265    2585 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325268    2585 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325273    2585 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325276    2585 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325279    2585 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325283    2585 flags.go:64] FLAG: --logging-format="text"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325286    2585 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325289    2585 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325292    2585 flags.go:64] FLAG: --manifest-url=""
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325295    2585 flags.go:64] FLAG: --manifest-url-header=""
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325299    2585 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325302    2585 flags.go:64] FLAG: --max-open-files="1000000"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325306    2585 flags.go:64] FLAG: --max-pods="110"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325309    2585 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325312    2585 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325315    2585 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325318    2585 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325320    2585 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325323    2585 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325330    2585 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325337    2585 flags.go:64] FLAG: --node-status-max-images="50"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325340    2585 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325344    2585 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 01 17:15:37.604912 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325347    2585 flags.go:64] FLAG: --pod-cidr=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325350    2585 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325355    2585 flags.go:64] FLAG: --pod-manifest-path=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325357    2585 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325360    2585 flags.go:64] FLAG: --pods-per-core="0"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325363    2585 flags.go:64] FLAG: --port="10250"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325366    2585 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325370    2585 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ff4a026393fee92c"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325373    2585 flags.go:64] FLAG: --qos-reserved=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325376    2585 flags.go:64] FLAG: --read-only-port="10255"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325379    2585 flags.go:64] FLAG: --register-node="true"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325382    2585 flags.go:64] FLAG: --register-with-taints=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325386    2585 flags.go:64] FLAG: --registry-burst="10"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325388    2585 flags.go:64] FLAG: --registry-qps="5"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325392    2585 flags.go:64] FLAG: --reserved-cpus=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325395    2585 flags.go:64] FLAG: --reserved-memory=""
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325399    2585 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325402    2585 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325405    2585 flags.go:64] FLAG: --rotate-certificates="false"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325408    2585 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325410    2585 flags.go:64] FLAG: --runonce="false"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325413    2585 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325416    2585 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325419    2585 flags.go:64] FLAG: --seccomp-default="false"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325422    2585 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 01 17:15:37.605633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325425    2585 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325428    2585 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325431    2585 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325434    2585 flags.go:64] FLAG: --storage-driver-password="root"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325439    2585 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325442    2585 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325444    2585 flags.go:64] FLAG: --storage-driver-user="root"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325447    2585 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325450    2585 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325453    2585 flags.go:64] FLAG: --system-cgroups=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325456    2585 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325461    2585 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325464    2585 flags.go:64] FLAG: --tls-cert-file=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325467    2585 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325475    2585 flags.go:64] FLAG: --tls-min-version=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325479    2585 flags.go:64] FLAG: --tls-private-key-file=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325482    2585 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325485    2585 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325487    2585 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325490    2585 flags.go:64] FLAG: --v="2"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325494    2585 flags.go:64] FLAG: --version="false"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325499    2585 flags.go:64] FLAG: --vmodule=""
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325503    2585 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.325506    2585 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325643    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.606397 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325646    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325649    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325651    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325655    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325657    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325660    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325662    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325665    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325668    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325670    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325674    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325679    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325681    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325684    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325687    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325689    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325692    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325695    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325697    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325700    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.607253 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325702    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325705    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325707    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325711    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325713    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325716    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325718    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325721    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325725    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325727    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325730    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325733    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325736    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325738    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325741    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325743    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325746    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325748    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325751    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.607837 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325753    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325756    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325759    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325761    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325765    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325768    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325770    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325773    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325776    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325778    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325781    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325783    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325786    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325788    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325791    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325794    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325798    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325802    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325805    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325808    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.608521 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325810    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325814    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325817    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325820    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325822    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325825    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325827    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325830    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325833    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325835    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325838    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325840    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325843    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325845    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325848    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325850    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325854    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325857    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325859    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325862    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.609120 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325864    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325868    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325871    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325874    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325877    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325880    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325882    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325885    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325888    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325890    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325893    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325896    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325898    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325902    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325905    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325907    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325910    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325912    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.724832 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.325915    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.326716    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.333582    2585 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.333601    2585 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.333616    2585 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.333623    2585 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333682    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333688    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333692    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333695    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333698    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333701    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333704    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333707    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333710    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.725644 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333712    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333715    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333718    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333720    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333723    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333726    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333729    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333732    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333734    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333737    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333740    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333743    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333745    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333748    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333751    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333754    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333757    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333759    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333762    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333765    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.726472 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333769    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333773    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333777    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333779    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333782    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333784    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333787    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333790    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333792    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333795    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333798    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333801    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333804    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333807    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333811    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333814    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333817    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333823    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333826    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.727064 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333829    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333832    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333837    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333839    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333842    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333845    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333848    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333851    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333853    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333856    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333858    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333861    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333864    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333866    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333869    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333872    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333875    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333877    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333880    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.727598 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333882    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333886    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333890    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333893    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333895    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333898    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333901    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333904    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333906    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333909    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333912    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333917    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333920    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333922    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333925    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333929    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333932    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333934    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333937    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.728150 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333940    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333942    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333944    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333947    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333950    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333952    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333955    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333957    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333960    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333963    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333966    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333968    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.333971    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.333976    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334116    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 01 17:15:37.728679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334122    2585 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334125    2585 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334128    2585 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334131    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334134    2585 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334137    2585 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334139    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334141    2585 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334144    2585 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334146    2585 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334151    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334153    2585 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334156    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334160    2585 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334163    2585 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334165    2585 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334168    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334186    2585 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334189    2585 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334192    2585 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 01 17:15:37.729131 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334194    2585 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334197    2585 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334199    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334203    2585 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334208    2585 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334211    2585 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334214    2585 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334217    2585 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334220    2585 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334223    2585 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334226    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334228    2585 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334230    2585 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334233    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334236    2585 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334238    2585 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334240    2585 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334243    2585 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334246    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334248    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 01 17:15:37.729698 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334251    2585 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334254    2585 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334258    2585 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334260    2585 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334263    2585 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334267    2585 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334269    2585 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334272    2585 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334274    2585 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334277    2585 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334279    2585 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334282    2585 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334285    2585 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334287    2585 feature_gate.go:328] unrecognized feature gate: Example2
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334290    2585 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334292    2585 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334295    2585 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334297    2585 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334300    2585 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334302    2585 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 01 17:15:37.730348 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334306    2585 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334308    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334311    2585 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334314    2585 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334318    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334321    2585 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334324    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334327    2585 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334331    2585 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334334    2585 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334337    2585 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334340    2585 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334343    2585 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334345    2585 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334349    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334352    2585 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334355    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334359    2585 feature_gate.go:328] unrecognized feature gate: Example
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334362    2585 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 01 17:15:37.730905 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334364    2585 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334366    2585 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334369    2585 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334372    2585 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334374    2585 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334377    2585 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334379    2585 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334382    2585 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334384    2585 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334387    2585 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334390    2585 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334393    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334395    2585 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334398    2585 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334401    2585 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334404    2585 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334406    2585 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334409    2585 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 01 17:15:37.731468 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:37.334411    2585 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.334417    2585 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.334552    2585 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.336885    2585 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.338743    2585 server.go:1019] "Starting client certificate rotation"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.338859    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.339712    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.341562    2585 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.345408    2585 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.359633    2585 log.go:25] "Validated CRI v1 runtime API"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.361915    2585 log.go:25] "Validated CRI v1 image API"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.362736    2585 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 01 17:15:37.732037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.366432    2585 fs.go:135] Filesystem UUIDs: map[01a0b9be-4008-4b84-9691-f464fdc9e62e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 aa08e053-34d0-4e24-b290-1d2d572bb798:/dev/nvme0n1p4]
Jul 01 17:15:37.732435 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.366448    2585 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.372708    2585 manager.go:216] Machine: {Timestamp:2026-07-01 17:15:37.371192077 +0000 UTC m=+0.358039920 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098575 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2027094e91198dfb264649e83eff8c SystemUUID:ec202709-4e91-198d-fb26-4649e83eff8c BootID:968702a3-b0c9-4108-b510-6694437af577 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:de:7e:7b:10:4d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:de:7e:7b:10:4d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:22:81:c1:30:29 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.372815    2585 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.372903    2585 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.373873    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.376712    2585 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.376743    2585 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-93.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 01 17:15:37.732633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.378387    2585 topology_manager.go:138] "Creating topology manager with none policy"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.378399    2585 container_manager_linux.go:309] "Creating device plugin manager"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.378407    2585 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.379278    2585 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.380310    2585 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.383816    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.383951    2585 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.383985    2585 kubelet.go:486] "Attempting to sync node with API server"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384004    2585 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384019    2585 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384032    2585 kubelet.go:398] "Adding apiserver pod source"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384041    2585 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384374    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.384389    2585 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.390320    2585 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392038    2585 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392067    2585 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392897    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392910    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392917    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392923    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392931    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.392937    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393070    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393076    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393083    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393089    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393108    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.393116    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.395066    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.395076    2585 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.397109    2585 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-93.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.397965    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.397967    2585 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-93.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.398439    2585 server.go:1268] "Started kubelet"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.398524    2585 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.398585    2585 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.398647    2585 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.400002    2585 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.405834    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67v98"
Jul 01 17:15:37.732926 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.406248    2585 server.go:310] "Adding debug handlers to kubelet server"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.408099    2585 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.411391    2585 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.411843    2585 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.412566    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.412644    2585 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.412668    2585 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.412678    2585 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413292    2585 reconstruct.go:97] "Volume reconstruction finished"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413305    2585 reconciler.go:29] "Reconciler: start to sync state"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413382    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67v98"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413513    2585 factory.go:55] Registering systemd factory
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413529    2585 factory.go:223] Registration of the systemd container factory successfully
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413795    2585 factory.go:153] Registering CRI-O factory
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413812    2585 factory.go:223] Registration of the crio container factory successfully
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413863    2585 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413891    2585 factory.go:103] Registering Raw factory
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.413907    2585 manager.go:1196] Started watching for new ooms in manager
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.414304    2585 manager.go:349] Starting recovery of all containers
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.421466    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.424973    2585 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-93.ec2.internal\" not found" node="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.426375    2585 manager.go:354] Recovery completed
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.432290    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.435913    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.435951    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.435966    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.436675    2585 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.436684    2585 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.436705    2585 state_mem.go:36] "Initialized new in-memory state store"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.438043    2585 policy_none.go:49] "None policy: Start"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.438056    2585 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.438067    2585 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.439258    2585 policy_none.go:47] "Start"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.440001    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.441107    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.441130    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.441147    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.473980    2585 manager.go:337] "Starting Device Plugin manager"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.474234    2585 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474250    2585 server.go:87] "Starting device plugin registration server"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474533    2585 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474545    2585 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474746    2585 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474833    2585 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 01 17:15:37.734065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.474843    2585 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.475649    2585 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.475707    2585 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.505743    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.506999    2585 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.507035    2585 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.507059    2585 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.507093    2585 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.509848    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.575343    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.576460    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.576494    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.576505    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.576528    2585 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.588239    2585 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.588266    2585 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-93.ec2.internal\": node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.607757    2585 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"]
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.608481    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.609971    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.610002    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.610031    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.611404    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.611594    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.611619    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612310    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612331    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612343    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612570    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612585    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.612597    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.613747    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.613902    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.613928    2585 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 01 17:15:37.735316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614500    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ca5b1431b790f3e800f6e3e0ef9703f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-93.ec2.internal\" (UID: \"3ca5b1431b790f3e800f6e3e0ef9703f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614525    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614549    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614836    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientMemory"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614855    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasNoDiskPressure"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.614868    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeHasSufficientPID"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.639811    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-93.ec2.internal\" not found" node="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.644266    2585 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-93.ec2.internal\" not found" node="ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.714186    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715284    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ca5b1431b790f3e800f6e3e0ef9703f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-93.ec2.internal\" (UID: \"3ca5b1431b790f3e800f6e3e0ef9703f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715305    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715404    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715410    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3ca5b1431b790f3e800f6e3e0ef9703f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-93.ec2.internal\" (UID: \"3ca5b1431b790f3e800f6e3e0ef9703f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715458    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.736255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.715512    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d58cbdfcde03c9238601c3631492c4e7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal\" (UID: \"d58cbdfcde03c9238601c3631492c4e7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:37.893240 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.814943    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.043594 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:37.915730    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.043594 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.944971    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:38.043594 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:37.948203    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:38.043594 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:38.016036    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.200499 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:38.116491    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.358801 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:38.216954    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.358801 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:38.317489    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.358801 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.338724    2585 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 01 17:15:38.358801 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.338881    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 17:15:38.358801 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.338915    2585 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 01 17:15:38.489517 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:38.418161    2585 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-93.ec2.internal\" not found"
Jul 01 17:15:38.489517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.418187    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 17:10:38 +0000 UTC" deadline="2028-03-09 03:00:37.738238184 +0000 UTC"
Jul 01 17:15:38.489517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.418208    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14793h44m59.320032963s"
Jul 01 17:15:38.489517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.453005    2585 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:38.494411 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.494026    2585 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:38.569279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.512934    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:38.569279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.524552    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 17:15:38.569279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.525692    2585 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal"
Jul 01 17:15:38.569279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.531928    2585 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 01 17:15:38.924656 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:38.924616    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca5b1431b790f3e800f6e3e0ef9703f.slice/crio-4feee6fa9476522b919b8f7fbfb546535276bb22f8fb50e33b765423c6bfa178 WatchSource:0}: Error finding container 4feee6fa9476522b919b8f7fbfb546535276bb22f8fb50e33b765423c6bfa178: Status 404 returned error can't find the container with id 4feee6fa9476522b919b8f7fbfb546535276bb22f8fb50e33b765423c6bfa178
Jul 01 17:15:38.925002 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:38.924982    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58cbdfcde03c9238601c3631492c4e7.slice/crio-3f1b5812707dc87bb0baeefd2c373155e4cbcfef8bada19e679a36118cb4c82f WatchSource:0}: Error finding container 3f1b5812707dc87bb0baeefd2c373155e4cbcfef8bada19e679a36118cb4c82f: Status 404 returned error can't find the container with id 3f1b5812707dc87bb0baeefd2c373155e4cbcfef8bada19e679a36118cb4c82f
Jul 01 17:15:38.930140 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:38.930125    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 17:15:40.287975 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.287946    2585 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:40.419394 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.419347    2585 apiserver.go:52] "Watching apiserver"
Jul 01 17:15:40.429126 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.429100    2585 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 01 17:15:40.429507 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.429480    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h","openshift-cluster-node-tuning-operator/tuned-6vxmz","openshift-dns/node-resolver-fgjrz","openshift-network-diagnostics/network-check-target-kbtzf","openshift-network-operator/iptables-alerter-qzsj9","openshift-ovn-kubernetes/ovnkube-node-qgf8j","kube-system/global-pull-secret-syncer-wqwfq","kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal","openshift-image-registry/node-ca-mw5ct","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal","openshift-multus/multus-additional-cni-plugins-7tvs4","openshift-multus/multus-fhj6v","openshift-multus/network-metrics-daemon-2724j","kube-system/konnectivity-agent-8kvfx"]
Jul 01 17:15:40.431923 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.431901    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.432022 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.431968    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:40.433255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.433234    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.433333 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.433316    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:40.434546 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.434531    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.435817 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.435798    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:40.435914 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.435899    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:40.437146 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.436959    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.437146 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.437108    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.437545 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.437510    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-55nkr\""
Jul 01 17:15:40.439448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.437833    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.439773 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.439749    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.440522 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.440501    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.440624 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.440546    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.440624 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.440561    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 01 17:15:40.440624 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.440611    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-47q82\""
Jul 01 17:15:40.441513 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.441495    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.442087 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.442074    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 01 17:15:40.442720 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.442696    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.442810 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.442773    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.442870 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.442854    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.443615 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443559    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 01 17:15:40.443615 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443570    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 01 17:15:40.443768 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443683    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qxl8n\""
Jul 01 17:15:40.443768 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443713    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 01 17:15:40.443925 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443910    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 01 17:15:40.443969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.443918    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pbzgr\""
Jul 01 17:15:40.444049 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444033    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.444259 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444245    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.444756 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444404    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.444942 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444924    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 01 17:15:40.445079 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444956    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.445079 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.444970    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.445471 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.445458    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gcgqx\""
Jul 01 17:15:40.445759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.445745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.446385 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.446369    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.446385 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.446381    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pxn68\""
Jul 01 17:15:40.446576 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.446564    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.447425 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.447411    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.447531 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.447513    2585 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 01 17:15:40.447815 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.447798    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7j995\""
Jul 01 17:15:40.448114 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.448081    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 01 17:15:40.448192 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.448160    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 01 17:15:40.448876 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.448855    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.449731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.449714    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 01 17:15:40.449851 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.449835    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nfqvz\""
Jul 01 17:15:40.449961 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.449946    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 01 17:15:40.450035 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.449976    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 01 17:15:40.450107 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.450061    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 01 17:15:40.451171 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.451154    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 01 17:15:40.451350 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.451338    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 01 17:15:40.451403 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.451395    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-hbk4m\""
Jul 01 17:15:40.457969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.457949    2585 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 01 17:15:40.459160 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459142    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-k8s-cni-cncf-io\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459236 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459166    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.459236 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459184    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-sys-fs\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.459236 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459199    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459245    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-sys\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459289    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4832d76a-9f98-422b-adc6-e44703c5248f-host\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459310    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-hostroot\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459359    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-etc-kubernetes\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459391    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-device-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459420    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-system-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459479 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459466    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cnibin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459524    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cni-binary-copy\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459554    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-socket-dir-parent\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459580    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-kubelet\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459619    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-env-overrides\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459644    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-systemd\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459666    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-run\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459745    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-konnectivity-ca\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459769    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-os-release\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459793    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-netns\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459829    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-daemon-config\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.459865 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459853    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-node-log\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459876    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgxg9\" (UniqueName: \"kubernetes.io/projected/af69b820-575a-43d2-89da-70f548839e4f-kube-api-access-xgxg9\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-var-lib-kubelet\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459943    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.459967    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-host-slash\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460005    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxcxb\" (UniqueName: \"kubernetes.io/projected/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-kube-api-access-qxcxb\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460030    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vrs4\" (UniqueName: \"kubernetes.io/projected/4832d76a-9f98-422b-adc6-e44703c5248f-kube-api-access-6vrs4\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460054    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-multus\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460079    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af69b820-575a-43d2-89da-70f548839e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460130    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhvg9\" (UniqueName: \"kubernetes.io/projected/b1f7031e-8809-41a2-b19d-3f403e8e1942-kube-api-access-nhvg9\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460193    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-bin\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460250    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-script-lib\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460282    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-multus-certs\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460307    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btg7s\" (UniqueName: \"kubernetes.io/projected/ba2d162e-5b36-4378-a440-669ffe4fc9d4-kube-api-access-btg7s\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460331    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-socket-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460354    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-kubernetes\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460377    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460397    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-hosts-file\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460426    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv45d\" (UniqueName: \"kubernetes.io/projected/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-kube-api-access-vv45d\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460466    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-etc-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460490    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-host\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460510    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-etc-tuned\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460524    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-conf-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460552    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-log-socket\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460580    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-etc-selinux\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460615    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-iptables-alerter-script\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.460867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460643    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w24kt\" (UniqueName: \"kubernetes.io/projected/7c07131d-6f49-44bf-acd6-d41b3bb662eb-kube-api-access-w24kt\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460664    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-var-lib-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460685    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460703    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460723    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-modprobe-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460746    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-cnibin\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460779    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4832d76a-9f98-422b-adc6-e44703c5248f-serviceca\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460814    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-agent-certs\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460843    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460864    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-systemd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460882    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-registration-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460905    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwm98\" (UniqueName: \"kubernetes.io/projected/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kube-api-access-kwm98\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460924    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-netd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460947    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-config\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460971    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.460994    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-kubelet\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461010    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-slash\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461028    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-ovn\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461078    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461095    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-conf\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461118    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-bin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461153    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysconfig\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461176    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-tmp\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461190    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461204    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-systemd-units\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461218    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-lib-modules\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.461977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461233    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461258    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-dbus\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461271    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-tmp-dir\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461285    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-netns\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461301    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2x2\" (UniqueName: \"kubernetes.io/projected/09d0c844-03fc-4632-ae14-28b3e1097701-kube-api-access-6h2x2\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461328    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-system-cni-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461346    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-os-release\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.462358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.461365    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-kubelet-config\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.482251 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.482229    2585 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvg98"
Jul 01 17:15:40.490082 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.490061    2585 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvg98"
Jul 01 17:15:40.545578 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.545470    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal" event={"ID":"3ca5b1431b790f3e800f6e3e0ef9703f","Type":"ContainerStarted","Data":"4feee6fa9476522b919b8f7fbfb546535276bb22f8fb50e33b765423c6bfa178"}
Jul 01 17:15:40.546511 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.546485    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal" event={"ID":"d58cbdfcde03c9238601c3631492c4e7","Type":"ContainerStarted","Data":"3f1b5812707dc87bb0baeefd2c373155e4cbcfef8bada19e679a36118cb4c82f"}
Jul 01 17:15:40.547823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.547807    2585 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 01 17:15:40.561937 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.561912    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-system-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.561946    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cnibin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.561969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cni-binary-copy\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.561990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-socket-dir-parent\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-kubelet\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562032    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cnibin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562034    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-system-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562080    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-kubelet\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562079    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-env-overrides\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562109    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-systemd\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-run\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-konnectivity-ca\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562162    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-os-release\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562139    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-socket-dir-parent\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562189    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-netns\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562187    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-systemd\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562201    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-run\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562311    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-netns\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562330    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-os-release\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-daemon-config\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562372    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-node-log\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562414    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgxg9\" (UniqueName: \"kubernetes.io/projected/af69b820-575a-43d2-89da-70f548839e4f-kube-api-access-xgxg9\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562443    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-var-lib-kubelet\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562469    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562494    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-host-slash\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562516    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxcxb\" (UniqueName: \"kubernetes.io/projected/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-kube-api-access-qxcxb\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562541    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vrs4\" (UniqueName: \"kubernetes.io/projected/4832d76a-9f98-422b-adc6-e44703c5248f-kube-api-access-6vrs4\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-multus\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562610    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af69b820-575a-43d2-89da-70f548839e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562642    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562670    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhvg9\" (UniqueName: \"kubernetes.io/projected/b1f7031e-8809-41a2-b19d-3f403e8e1942-kube-api-access-nhvg9\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562692    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-host-slash\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562692    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-cni-binary-copy\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.562952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562742    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-konnectivity-ca\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562778    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-node-log\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562644    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-env-overrides\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562832    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-daemon-config\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562695    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-bin\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562876    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-var-lib-kubelet\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562882    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-script-lib\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562916    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-multus-certs\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562959    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btg7s\" (UniqueName: \"kubernetes.io/projected/ba2d162e-5b36-4378-a440-669ffe4fc9d4-kube-api-access-btg7s\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562980    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-multus-certs\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.562987    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-socket-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563139    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-socket-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563136    2585 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563177    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-kubernetes\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.563734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563203    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-bin\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563237    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-multus\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563250    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-kubernetes\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563275    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-hosts-file\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.563283    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563300    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv45d\" (UniqueName: \"kubernetes.io/projected/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-kube-api-access-vv45d\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563322    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-etc-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.563341    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:41.063320894 +0000 UTC m=+3.015747422 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563344    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563358    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-etc-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563365    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-host\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563360    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-script-lib\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563378    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563389    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-etc-tuned\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.564506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563405    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-hosts-file\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563413    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-conf-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563453    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-host\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563461    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-log-socket\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563470    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-conf-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563491    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-etc-selinux\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563505    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-log-socket\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563519    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-iptables-alerter-script\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563543    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w24kt\" (UniqueName: \"kubernetes.io/projected/7c07131d-6f49-44bf-acd6-d41b3bb662eb-kube-api-access-w24kt\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563564    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-var-lib-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563608    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563613    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-etc-selinux\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563653    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563682    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563710    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-modprobe-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563729    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-var-lib-openvswitch\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563734    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-cnibin\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4832d76a-9f98-422b-adc6-e44703c5248f-serviceca\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563766    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563802    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-cnibin\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563825    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-modprobe-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563829    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-agent-certs\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563859    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563884    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-systemd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563908    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-registration-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563937    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwm98\" (UniqueName: \"kubernetes.io/projected/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kube-api-access-kwm98\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563966    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-netd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563983    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-iptables-alerter-script\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.565959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.563990    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-config\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564038    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564048    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-registration-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564063    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-kubelet\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564089    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-systemd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564113    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-slash\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564138    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-ovn\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564163    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564187    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-conf\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564214    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-bin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564215    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4832d76a-9f98-422b-adc6-e44703c5248f-serviceca\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564004    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-multus-cni-dir\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564243    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-cni-netd\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564239    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysconfig\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564284    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-run-ovn\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.566573 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564285    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysconfig\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564312    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-tmp\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564320    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564356    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-kubelet\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.564392    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.564453    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:15:41.064423613 +0000 UTC m=+3.016850140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564502    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564504    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-conf\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564393    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-slash\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564534    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/af69b820-575a-43d2-89da-70f548839e4f-ovnkube-config\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564540    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-systemd-units\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564509    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-var-lib-cni-bin\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564570    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-lib-modules\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564610    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-dbus\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564658    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-tmp-dir\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564612    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-systemd-units\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564681    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-netns\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564698    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-lib-modules\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564706    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2x2\" (UniqueName: \"kubernetes.io/projected/09d0c844-03fc-4632-ae14-28b3e1097701-kube-api-access-6h2x2\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564739    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-dbus\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-system-cni-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564763    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/af69b820-575a-43d2-89da-70f548839e4f-host-run-netns\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564769    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564781    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-os-release\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564781    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-system-cni-dir\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564828    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b1f7031e-8809-41a2-b19d-3f403e8e1942-os-release\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564879    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-kubelet-config\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.567893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564902    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-k8s-cni-cncf-io\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564936    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0496a8d7-d920-4aea-987a-1903d992115b-kubelet-config\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564948    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564975    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-sys-fs\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564978    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-tmp-dir\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.564997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565031    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-sys\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565054    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565064    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-sys-fs\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565068    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-host-run-k8s-cni-cncf-io\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565064    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565109    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4832d76a-9f98-422b-adc6-e44703c5248f-host\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565120    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-sys\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565134    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-hostroot\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565142    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/09d0c844-03fc-4632-ae14-28b3e1097701-etc-sysctl-d\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.568499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565157    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4832d76a-9f98-422b-adc6-e44703c5248f-host\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565164    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-etc-kubernetes\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565184    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-hostroot\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565187    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-device-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565264    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c07131d-6f49-44bf-acd6-d41b3bb662eb-etc-kubernetes\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.565224    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/992e8420-1e1f-4ad3-a2aa-79364c5b0028-device-dir\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.566282    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b1f7031e-8809-41a2-b19d-3f403e8e1942-cni-binary-copy\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.567556    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-tmp\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.567585    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/09d0c844-03fc-4632-ae14-28b3e1097701-etc-tuned\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.569156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.567797    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/af69b820-575a-43d2-89da-70f548839e4f-ovn-node-metrics-cert\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.570359 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.570339    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc350dd5-2f49-4d7e-b36b-71bec39ea05a-agent-certs\") pod \"konnectivity-agent-8kvfx\" (UID: \"fc350dd5-2f49-4d7e-b36b-71bec39ea05a\") " pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.577503 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.577478    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vrs4\" (UniqueName: \"kubernetes.io/projected/4832d76a-9f98-422b-adc6-e44703c5248f-kube-api-access-6vrs4\") pod \"node-ca-mw5ct\" (UID: \"4832d76a-9f98-422b-adc6-e44703c5248f\") " pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.578294 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.578272    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:40.578740 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.578300    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:40.578740 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.578331    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:40.578740 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:40.578454    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:41.078433301 +0000 UTC m=+3.030859844 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:40.580582 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.580555    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxcxb\" (UniqueName: \"kubernetes.io/projected/1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564-kube-api-access-qxcxb\") pod \"iptables-alerter-qzsj9\" (UID: \"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564\") " pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.580710 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.580690    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgxg9\" (UniqueName: \"kubernetes.io/projected/af69b820-575a-43d2-89da-70f548839e4f-kube-api-access-xgxg9\") pod \"ovnkube-node-qgf8j\" (UID: \"af69b820-575a-43d2-89da-70f548839e4f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.580782 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.580732    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv45d\" (UniqueName: \"kubernetes.io/projected/fe4d7fa6-f0ff-435d-98b8-80e993e6a414-kube-api-access-vv45d\") pod \"node-resolver-fgjrz\" (UID: \"fe4d7fa6-f0ff-435d-98b8-80e993e6a414\") " pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.580830 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.580796    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwm98\" (UniqueName: \"kubernetes.io/projected/992e8420-1e1f-4ad3-a2aa-79364c5b0028-kube-api-access-kwm98\") pod \"aws-ebs-csi-driver-node-jbp9h\" (UID: \"992e8420-1e1f-4ad3-a2aa-79364c5b0028\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.581016 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.580992    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btg7s\" (UniqueName: \"kubernetes.io/projected/ba2d162e-5b36-4378-a440-669ffe4fc9d4-kube-api-access-btg7s\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:40.581329 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.581303    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhvg9\" (UniqueName: \"kubernetes.io/projected/b1f7031e-8809-41a2-b19d-3f403e8e1942-kube-api-access-nhvg9\") pod \"multus-additional-cni-plugins-7tvs4\" (UID: \"b1f7031e-8809-41a2-b19d-3f403e8e1942\") " pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.581867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.581848    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w24kt\" (UniqueName: \"kubernetes.io/projected/7c07131d-6f49-44bf-acd6-d41b3bb662eb-kube-api-access-w24kt\") pod \"multus-fhj6v\" (UID: \"7c07131d-6f49-44bf-acd6-d41b3bb662eb\") " pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.582170 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.582138    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2x2\" (UniqueName: \"kubernetes.io/projected/09d0c844-03fc-4632-ae14-28b3e1097701-kube-api-access-6h2x2\") pod \"tuned-6vxmz\" (UID: \"09d0c844-03fc-4632-ae14-28b3e1097701\") " pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.723308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.723277    2585 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 01 17:15:40.748700 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.748534    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fgjrz"
Jul 01 17:15:40.755759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.755731    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qzsj9"
Jul 01 17:15:40.758180 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.758125    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4d7fa6_f0ff_435d_98b8_80e993e6a414.slice/crio-4f612c5279f8e71c5863718a9ea2c034ba3c84eb262e986fad0c9870baa2290c WatchSource:0}: Error finding container 4f612c5279f8e71c5863718a9ea2c034ba3c84eb262e986fad0c9870baa2290c: Status 404 returned error can't find the container with id 4f612c5279f8e71c5863718a9ea2c034ba3c84eb262e986fad0c9870baa2290c
Jul 01 17:15:40.763936 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.763861    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fhj6v"
Jul 01 17:15:40.770254 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.770233    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:15:40.771445 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.771361    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b3aaaf6_18d1_4929_b45f_2d3b5c6e6564.slice/crio-fa73fa1f5d05b717ea061420860e5cce77f8f52134c4046b8c58f2e655b7543f WatchSource:0}: Error finding container fa73fa1f5d05b717ea061420860e5cce77f8f52134c4046b8c58f2e655b7543f: Status 404 returned error can't find the container with id fa73fa1f5d05b717ea061420860e5cce77f8f52134c4046b8c58f2e655b7543f
Jul 01 17:15:40.775031 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.775013    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mw5ct"
Jul 01 17:15:40.782189 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.782025    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h"
Jul 01 17:15:40.783449 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.783422    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c07131d_6f49_44bf_acd6_d41b3bb662eb.slice/crio-dbd91cac14f45489614760d8f5d443078bb8e5a5a90e775683b240db04862ec1 WatchSource:0}: Error finding container dbd91cac14f45489614760d8f5d443078bb8e5a5a90e775683b240db04862ec1: Status 404 returned error can't find the container with id dbd91cac14f45489614760d8f5d443078bb8e5a5a90e775683b240db04862ec1
Jul 01 17:15:40.785439 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.785418    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6vxmz"
Jul 01 17:15:40.789582 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.789557    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf69b820_575a_43d2_89da_70f548839e4f.slice/crio-45a38e3d88e04792686eda51dddeec06521e3852527e7c8646a2532e667456a3 WatchSource:0}: Error finding container 45a38e3d88e04792686eda51dddeec06521e3852527e7c8646a2532e667456a3: Status 404 returned error can't find the container with id 45a38e3d88e04792686eda51dddeec06521e3852527e7c8646a2532e667456a3
Jul 01 17:15:40.790555 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.790519    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:15:40.793813 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:40.793796    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7tvs4"
Jul 01 17:15:40.797615 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.797479    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod992e8420_1e1f_4ad3_a2aa_79364c5b0028.slice/crio-a677a6ab0d2798dbdba7cca2d6fe8590e3a9054dcfd81b2cdd821f2bf507f832 WatchSource:0}: Error finding container a677a6ab0d2798dbdba7cca2d6fe8590e3a9054dcfd81b2cdd821f2bf507f832: Status 404 returned error can't find the container with id a677a6ab0d2798dbdba7cca2d6fe8590e3a9054dcfd81b2cdd821f2bf507f832
Jul 01 17:15:40.797911 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.797853    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4832d76a_9f98_422b_adc6_e44703c5248f.slice/crio-0775944ab8bbed081e54af53a23efd4382e902710cf06bb77e777bf6d5335e30 WatchSource:0}: Error finding container 0775944ab8bbed081e54af53a23efd4382e902710cf06bb77e777bf6d5335e30: Status 404 returned error can't find the container with id 0775944ab8bbed081e54af53a23efd4382e902710cf06bb77e777bf6d5335e30
Jul 01 17:15:40.808917 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.808745    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc350dd5_2f49_4d7e_b36b_71bec39ea05a.slice/crio-c15c9c29107d0d3f90a2ed11195c11a5a8dd0cb675697f9eca48ac1230edebe2 WatchSource:0}: Error finding container c15c9c29107d0d3f90a2ed11195c11a5a8dd0cb675697f9eca48ac1230edebe2: Status 404 returned error can't find the container with id c15c9c29107d0d3f90a2ed11195c11a5a8dd0cb675697f9eca48ac1230edebe2
Jul 01 17:15:40.810331 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.810158    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d0c844_03fc_4632_ae14_28b3e1097701.slice/crio-980e08083f52792c9bc82029e04122a3cf945d8a8b0bc86f296ecb16d53c5496 WatchSource:0}: Error finding container 980e08083f52792c9bc82029e04122a3cf945d8a8b0bc86f296ecb16d53c5496: Status 404 returned error can't find the container with id 980e08083f52792c9bc82029e04122a3cf945d8a8b0bc86f296ecb16d53c5496
Jul 01 17:15:40.817317 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:15:40.817225    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f7031e_8809_41a2_b19d_3f403e8e1942.slice/crio-4804c6af852db76218b9d8d57a40453932294e151e5c2c421188ed7db20588fd WatchSource:0}: Error finding container 4804c6af852db76218b9d8d57a40453932294e151e5c2c421188ed7db20588fd: Status 404 returned error can't find the container with id 4804c6af852db76218b9d8d57a40453932294e151e5c2c421188ed7db20588fd
Jul 01 17:15:41.070150 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.070066    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:41.070150 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.070134    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:41.070343 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.070217    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:41.070343 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.070284    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:41.070343 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.070298    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:42.070276753 +0000 UTC m=+4.022703297 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:41.070343 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.070340    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:15:42.070321659 +0000 UTC m=+4.022748205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:41.170941 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.170901    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:41.171179 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.171120    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:41.171179 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.171142    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:41.171179 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.171159    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:41.171357 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:41.171250    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:42.171227884 +0000 UTC m=+4.123654424 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:41.492874 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.492806    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 17:10:40 +0000 UTC" deadline="2027-12-16 13:30:04.429925802 +0000 UTC"
Jul 01 17:15:41.492874 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.492867    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12788h14m22.937062335s"
Jul 01 17:15:41.551140 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.551098    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mw5ct" event={"ID":"4832d76a-9f98-422b-adc6-e44703c5248f","Type":"ContainerStarted","Data":"0775944ab8bbed081e54af53a23efd4382e902710cf06bb77e777bf6d5335e30"}
Jul 01 17:15:41.553080 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.553050    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"45a38e3d88e04792686eda51dddeec06521e3852527e7c8646a2532e667456a3"}
Jul 01 17:15:41.554661 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.554631    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fgjrz" event={"ID":"fe4d7fa6-f0ff-435d-98b8-80e993e6a414","Type":"ContainerStarted","Data":"4f612c5279f8e71c5863718a9ea2c034ba3c84eb262e986fad0c9870baa2290c"}
Jul 01 17:15:41.555899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.555872    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerStarted","Data":"4804c6af852db76218b9d8d57a40453932294e151e5c2c421188ed7db20588fd"}
Jul 01 17:15:41.557357 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.557333    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vxmz" event={"ID":"09d0c844-03fc-4632-ae14-28b3e1097701","Type":"ContainerStarted","Data":"980e08083f52792c9bc82029e04122a3cf945d8a8b0bc86f296ecb16d53c5496"}
Jul 01 17:15:41.559928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.559906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kvfx" event={"ID":"fc350dd5-2f49-4d7e-b36b-71bec39ea05a","Type":"ContainerStarted","Data":"c15c9c29107d0d3f90a2ed11195c11a5a8dd0cb675697f9eca48ac1230edebe2"}
Jul 01 17:15:41.562516 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.562493    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h" event={"ID":"992e8420-1e1f-4ad3-a2aa-79364c5b0028","Type":"ContainerStarted","Data":"a677a6ab0d2798dbdba7cca2d6fe8590e3a9054dcfd81b2cdd821f2bf507f832"}
Jul 01 17:15:41.564029 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.563966    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fhj6v" event={"ID":"7c07131d-6f49-44bf-acd6-d41b3bb662eb","Type":"ContainerStarted","Data":"dbd91cac14f45489614760d8f5d443078bb8e5a5a90e775683b240db04862ec1"}
Jul 01 17:15:41.565549 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:41.565525    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qzsj9" event={"ID":"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564","Type":"ContainerStarted","Data":"fa73fa1f5d05b717ea061420860e5cce77f8f52134c4046b8c58f2e655b7543f"}
Jul 01 17:15:42.079151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.079028    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:42.079151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.079145    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:42.079496 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.079281    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:42.079496 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.079348    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:44.079326097 +0000 UTC m=+6.031752634 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:42.079902 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.079874    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:42.079992 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.079983    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:15:44.079966145 +0000 UTC m=+6.032392676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:42.180913 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.180240    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:42.180913 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.180456    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:42.180913 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.180475    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:42.180913 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.180488    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:42.180913 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.180551    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:44.180533137 +0000 UTC m=+6.132959685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:42.496714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.496455    2585 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 17:10:40 +0000 UTC" deadline="2027-11-27 17:04:44.561602881 +0000 UTC"
Jul 01 17:15:42.496714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.496496    2585 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12335h49m2.065111812s"
Jul 01 17:15:42.545517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.545481    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:42.545716 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.545632    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:42.545798 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.545730    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:42.545846 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.545811    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:42.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:42.545890    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:42.545996 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:42.545977    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:43.584743 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:43.583623    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal" event={"ID":"3ca5b1431b790f3e800f6e3e0ef9703f","Type":"ContainerStarted","Data":"5112d0b1783031d951cada7f418cd5ca9e7de908381e49791a124498d0ea8775"}
Jul 01 17:15:44.097367 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.097328    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:44.097632 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.097440    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:44.109288 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.107819    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:44.109288 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.107844    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:44.109288 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.107925    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:15:48.107902388 +0000 UTC m=+10.060328926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:44.109288 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.107949    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:48.107937354 +0000 UTC m=+10.060363881 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:44.198306 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.198264    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:44.198503 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.198419    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:44.198503 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.198442    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:44.198503 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.198455    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:44.198723 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.198532    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:48.198512759 +0000 UTC m=+10.150939296 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.541746    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.541894    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.542297    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.542389    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.542450    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:44.544863 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:44.542515    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:44.590768 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.590689    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal" event={"ID":"d58cbdfcde03c9238601c3631492c4e7","Type":"ContainerStarted","Data":"24c0802d3156fb3dedb387b748d934811da2e69a7c41c659842249f00c9bc044"}
Jul 01 17:15:44.607499 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:44.606395    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-93.ec2.internal" podStartSLOduration=5.606373382 podStartE2EDuration="5.606373382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:15:43.599635839 +0000 UTC m=+5.552062387" watchObservedRunningTime="2026-07-01 17:15:44.606373382 +0000 UTC m=+6.558800027"
Jul 01 17:15:46.542451 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:46.542364    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:46.542935 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:46.542365    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:46.542935 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:46.542528    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:46.542935 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:46.542645    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:46.542935 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:46.542688    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:46.542935 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:46.542744    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:46.595843 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:46.595718    2585 generic.go:356] "Generic (PLEG): container finished" podID="d58cbdfcde03c9238601c3631492c4e7" containerID="24c0802d3156fb3dedb387b748d934811da2e69a7c41c659842249f00c9bc044" exitCode=0
Jul 01 17:15:46.595843 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:46.595778    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal" event={"ID":"d58cbdfcde03c9238601c3631492c4e7","Type":"ContainerDied","Data":"24c0802d3156fb3dedb387b748d934811da2e69a7c41c659842249f00c9bc044"}
Jul 01 17:15:48.130798 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.130745    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:48.131235 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.130854    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:48.131235 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.130935    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:48.131235 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.130965    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:48.131235 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.131017    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:15:56.130995243 +0000 UTC m=+18.083421785 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:48.131235 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.131040    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:56.131028993 +0000 UTC m=+18.083455521 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:48.231820 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.231765    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:48.232003 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.231976    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:48.232074 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.232007    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:48.232074 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.232023    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:48.232155 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.232096    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:15:56.2320774 +0000 UTC m=+18.184503927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:48.543515 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.543310    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:48.543515 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.543440    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:48.543778 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.543761    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:48.544026 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:48.543893    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:48.544026 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.543886    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:48.544026 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:48.543988    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:50.541894    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:50.541894    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:50.542042    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:50.542149    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:50.542207    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:50.542448 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:50.542264    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:52.542706 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:52.542667    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:52.543202 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:52.542667    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:52.543202 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:52.542810    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:52.543202 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:52.542667    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:52.543202 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:52.542896    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:52.543202 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:52.542970    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:54.542091 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:54.542055    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:54.542528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:54.542054    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:54.542528 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:54.542198    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:54.542528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:54.542055    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:54.542528 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:54.542308    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:54.542528 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:54.542434    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:56.195529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.195444    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:56.195529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.195498    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:56.196016 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.195624    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:56.196016 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.195684    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:16:12.195670121 +0000 UTC m=+34.148096646 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:15:56.196016 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.195624    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:56.196016 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.195748    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:12.195734903 +0000 UTC m=+34.148161443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:15:56.296604 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.296562    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:56.296801 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.296750    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:15:56.296801 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.296775    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:15:56.296801 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.296788    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:56.296945 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.296858    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:12.296837758 +0000 UTC m=+34.249264304 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:15:56.542643 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.542605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:56.542832 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.542605    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:56.542832 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:56.542749    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:56.542832 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.542745    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:56.542983 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.542867    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:15:56.542983 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:56.542952    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:58.542948 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:58.542908    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:15:58.543406 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:58.543045    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:15:58.543406 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:58.543085    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:15:58.543406 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:15:58.543147    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:15:58.543406 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:58.543264    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:15:58.543406 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:15:58.543345    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:00.542245 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.541939    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:00.542884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.541939    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:00.542884 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:00.542339    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:00.542884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.541995    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:00.542884 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:00.542420    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:00.542884 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:00.542488    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:00.625521 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.625493    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal" event={"ID":"d58cbdfcde03c9238601c3631492c4e7","Type":"ContainerStarted","Data":"c62bf8408204fa6b76f0e3e46e793136a70ad10bb579eb286aa1734633add1ae"}
Jul 01 17:16:00.627091 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.627062    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="42d514be8ee1864722234348992714bdc5e0db93bf58381b9290e796756d9b71" exitCode=0
Jul 01 17:16:00.627218 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.627101    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"42d514be8ee1864722234348992714bdc5e0db93bf58381b9290e796756d9b71"}
Jul 01 17:16:00.628483 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.628464    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6vxmz" event={"ID":"09d0c844-03fc-4632-ae14-28b3e1097701","Type":"ContainerStarted","Data":"d77694d88bb2cc863a653fbd8ddd9d444a8cd6a9c9e4cf9629edb61930663ecd"}
Jul 01 17:16:00.629987 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.629959    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8kvfx" event={"ID":"fc350dd5-2f49-4d7e-b36b-71bec39ea05a","Type":"ContainerStarted","Data":"ed4cb7d60290e917f959ada960f48f30578d140a598d30b61d9c6110ca922b2b"}
Jul 01 17:16:00.634063 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.634034    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h" event={"ID":"992e8420-1e1f-4ad3-a2aa-79364c5b0028","Type":"ContainerStarted","Data":"a678f1cdd3004c8bcec686ce87d8e66b0321301173e543bf0bfd3eeddadd45e8"}
Jul 01 17:16:00.635388 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.635365    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fhj6v" event={"ID":"7c07131d-6f49-44bf-acd6-d41b3bb662eb","Type":"ContainerStarted","Data":"63ba7dd9f753ece0adfa909484a5c28961d5899aded0212e5ec87dfc6d40712a"}
Jul 01 17:16:00.636809 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.636789    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mw5ct" event={"ID":"4832d76a-9f98-422b-adc6-e44703c5248f","Type":"ContainerStarted","Data":"c009788215943a65911a679fbf358ceb849e6eda3772fcb6dc174190c7fd5a0d"}
Jul 01 17:16:00.638764 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.638748    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:16:00.639103 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.639081    2585 generic.go:356] "Generic (PLEG): container finished" podID="af69b820-575a-43d2-89da-70f548839e4f" containerID="5eeb80a5dc72e2b90c42f90a6f8dd5dbfc37e6ff8bc8a6fe79d5c801a9bcdfde" exitCode=1
Jul 01 17:16:00.639163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.639143    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"e1bca695486591ec06441eae76bde01d1aea4b9554f7e2ea66f9f2d6ef670515"}
Jul 01 17:16:00.639236 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.639220    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"0233a2a9366407ab122cc6b89f996d909f319fd850b3462f93a20903fda61d9e"}
Jul 01 17:16:00.639291 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.639278    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerDied","Data":"5eeb80a5dc72e2b90c42f90a6f8dd5dbfc37e6ff8bc8a6fe79d5c801a9bcdfde"}
Jul 01 17:16:00.639344 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.639330    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"adcd411122be1dbfcdcd6e4a852386a02601c1a9dc45306b6489e7819a33afcf"}
Jul 01 17:16:00.640346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.640328    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fgjrz" event={"ID":"fe4d7fa6-f0ff-435d-98b8-80e993e6a414","Type":"ContainerStarted","Data":"59ada84dcc49d97be46ffe0e9093dfda011fe94cba2b2e96049d304307b8b141"}
Jul 01 17:16:00.643188 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.643153    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-93.ec2.internal" podStartSLOduration=21.643142192 podStartE2EDuration="21.643142192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.643104172 +0000 UTC m=+22.595530718" watchObservedRunningTime="2026-07-01 17:16:00.643142192 +0000 UTC m=+22.595568735"
Jul 01 17:16:00.656443 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.656405    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mw5ct" podStartSLOduration=3.423221341 podStartE2EDuration="22.65638169s" totalImagesPullingTime="19.233160349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.656215852 +0000 UTC m=+22.608642398" watchObservedRunningTime="2026-07-01 17:16:00.65638169 +0000 UTC m=+22.608808233"
Jul 01 17:16:00.671224 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.671182    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6vxmz" podStartSLOduration=3.4534926 podStartE2EDuration="22.671165953s" totalImagesPullingTime="19.217673353s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.670862622 +0000 UTC m=+22.623289168" watchObservedRunningTime="2026-07-01 17:16:00.671165953 +0000 UTC m=+22.623592564"
Jul 01 17:16:00.690847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.690801    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8kvfx" podStartSLOduration=3.467410627 podStartE2EDuration="22.690784412s" totalImagesPullingTime="19.223373785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.690747418 +0000 UTC m=+22.643173963" watchObservedRunningTime="2026-07-01 17:16:00.690784412 +0000 UTC m=+22.643210959"
Jul 01 17:16:00.733397 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.733351    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fhj6v" podStartSLOduration=3.4305276080000002 podStartE2EDuration="22.733334613s" totalImagesPullingTime="19.302807005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.732989631 +0000 UTC m=+22.685416178" watchObservedRunningTime="2026-07-01 17:16:00.733334613 +0000 UTC m=+22.685761158"
Jul 01 17:16:00.746993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:00.746943    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fgjrz" podStartSLOduration=3.472777232 podStartE2EDuration="22.746926392s" totalImagesPullingTime="19.27414916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:00.746646558 +0000 UTC m=+22.699073103" watchObservedRunningTime="2026-07-01 17:16:00.746926392 +0000 UTC m=+22.699352937"
Jul 01 17:16:01.645506 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.645432    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:16:01.645967 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.645838    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"6390b91291c536733f6b93152b2fca8d2d22413c5625eab47f8ded12aa912629"}
Jul 01 17:16:01.645967 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.645923    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"0a291eba909d489a71afe0cabf93143bbc7cd3e486c47e663eccca5002cd3a57"}
Jul 01 17:16:01.647244 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.647213    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qzsj9" event={"ID":"1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564","Type":"ContainerStarted","Data":"ccc3fd6db591088520e7d54ac5a46ae6a982ef35171d12e2408c811492d805da"}
Jul 01 17:16:01.663455 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.663406    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qzsj9" podStartSLOduration=4.407384406 podStartE2EDuration="23.663390067s" totalImagesPullingTime="19.256005661s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:01.663198336 +0000 UTC m=+23.615624882" watchObservedRunningTime="2026-07-01 17:16:01.663390067 +0000 UTC m=+23.615816613"
Jul 01 17:16:01.923721 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:01.923694    2585 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 01 17:16:02.521666 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.521544    2585 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T17:16:01.923715393Z","UUID":"a911eed7-429a-42f8-b6a9-be189d4cf2c9","Handler":null,"Name":"","Endpoint":""}
Jul 01 17:16:02.524792 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.524767    2585 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 01 17:16:02.524943 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.524806    2585 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 01 17:16:02.542233 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.542205    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:02.542400 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.542205    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:02.542400 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.542329    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:02.542400 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:02.542325    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:02.542555 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:02.542421    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:02.542632 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:02.542557    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:02.652341 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:02.652305    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h" event={"ID":"992e8420-1e1f-4ad3-a2aa-79364c5b0028","Type":"ContainerStarted","Data":"aed380c98c8f406f2452c17a1494f1927cfaa00f5a4804f62c2af3a03ed61c10"}
Jul 01 17:16:03.657964 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:03.657788    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:16:03.658372 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:03.658345    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"4ede0d21c278c5519cb9517a3165ba5f3c70896ba66019e9ef42a3a657b9f74a"}
Jul 01 17:16:03.729622 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:03.729576    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:16:03.730487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:03.730337    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:16:04.541757 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.541717    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:04.541989 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.541717    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:04.541989 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:04.541845    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:04.541989 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.541851    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:04.541989 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:04.541943    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:04.542208 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:04.542041    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:04.663257 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.663211    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h" event={"ID":"992e8420-1e1f-4ad3-a2aa-79364c5b0028","Type":"ContainerStarted","Data":"f7241d15aea721df864d8fe9052f2449b67ba2b5462bb61ab34e10b124f98624"}
Jul 01 17:16:04.663760 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.663742    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:16:04.664014 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.663990    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8kvfx"
Jul 01 17:16:04.681816 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:04.681761    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jbp9h" podStartSLOduration=4.237175624 podStartE2EDuration="26.68174445s" totalImagesPullingTime="22.444568826s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:04.68117808 +0000 UTC m=+26.633604627" watchObservedRunningTime="2026-07-01 17:16:04.68174445 +0000 UTC m=+26.634170995"
Jul 01 17:16:06.542759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.542573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:06.543571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.542573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:06.543571 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:06.542833    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:06.543571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.542627    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:06.543571 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:06.542897    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:06.543571 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:06.543002    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:06.669727 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.669695    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="143006201b96ee4401844fffddac8d67383d122c4643fe9c90346368f20dedf5" exitCode=0
Jul 01 17:16:06.669959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.669736    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"143006201b96ee4401844fffddac8d67383d122c4643fe9c90346368f20dedf5"}
Jul 01 17:16:06.673568 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.673549    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:16:06.673934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.673911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"e2ccc67bbd60668eb8f11243a83a753e7f1a04a21c6fcadf47d35d1cc3b34125"}
Jul 01 17:16:06.674375 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:06.674355    2585 scope.go:117] "RemoveContainer" containerID="5eeb80a5dc72e2b90c42f90a6f8dd5dbfc37e6ff8bc8a6fe79d5c801a9bcdfde"
Jul 01 17:16:07.681140 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.681112    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:16:07.681796 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.681520    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" event={"ID":"af69b820-575a-43d2-89da-70f548839e4f","Type":"ContainerStarted","Data":"a12d292fd161f85a322f1c543ba2fbcfa70d10d93ce70549ee2b809183f6199f"}
Jul 01 17:16:07.681967 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.681932    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:07.682112 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.682018    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:07.682112 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.682065    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:07.698744 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.698717    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:07.698898 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.698826    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:07.711107 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.711050    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j" podStartSLOduration=10.375301316 podStartE2EDuration="29.711029423s" totalImagesPullingTime="19.335728107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:07.709738578 +0000 UTC m=+29.662165144" watchObservedRunningTime="2026-07-01 17:16:07.711029423 +0000 UTC m=+29.663455971"
Jul 01 17:16:07.954665 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.954397    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kbtzf"]
Jul 01 17:16:07.954804 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.954782    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:07.954949 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:07.954914    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:07.955100 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.954965    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqwfq"]
Jul 01 17:16:07.955100 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.955050    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:07.955803 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:07.955126    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:07.955803 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.955608    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2724j"]
Jul 01 17:16:07.955803 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:07.955715    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:07.955923 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:07.955829    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:08.685528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:08.685493    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="1bec6dfb2fb948a27eb19256615f8f85cbd0b76bebec10a416e88f49c002d847" exitCode=0
Jul 01 17:16:08.686142 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:08.685568    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"1bec6dfb2fb948a27eb19256615f8f85cbd0b76bebec10a416e88f49c002d847"}
Jul 01 17:16:09.542201 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:09.542171    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:09.542386 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:09.542270    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:09.542386 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:09.542171    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:09.542386 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:09.542171    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:09.542386 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:09.542356    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:09.542527 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:09.542421    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:10.692969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:10.692929    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="b994239586d1fbf45660ea23ad5acb58adcb86bbcded4c7a4c364ccc2a7523c8" exitCode=0
Jul 01 17:16:10.693658 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:10.693004    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"b994239586d1fbf45660ea23ad5acb58adcb86bbcded4c7a4c364ccc2a7523c8"}
Jul 01 17:16:11.542623 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:11.542569    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:11.542826 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:11.542662    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:11.542826 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:11.542781    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:11.542939 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:11.542842    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:11.542988 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:11.542937    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:11.543045 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:11.543033    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:12.212910 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:12.212853    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:12.213390 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:12.212964    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:12.213390 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.213013    2585 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 01 17:16:12.213390 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.213062    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:16:12.213390 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.213088    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret podName:0496a8d7-d920-4aea-987a-1903d992115b nodeName:}" failed. No retries permitted until 2026-07-01 17:16:44.213067828 +0000 UTC m=+66.165494376 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret") pod "global-pull-secret-syncer-wqwfq" (UID: "0496a8d7-d920-4aea-987a-1903d992115b") : object "kube-system"/"original-pull-secret" not registered
Jul 01 17:16:12.213390 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.213114    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:44.213097384 +0000 UTC m=+66.165523925 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 01 17:16:12.313866 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:12.313828    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:12.314062 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.313996    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 01 17:16:12.314062 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.314023    2585 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 01 17:16:12.314062 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.314036    2585 projected.go:196] Error preparing data for projected volume kube-api-access-9996k for pod openshift-network-diagnostics/network-check-target-kbtzf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:16:12.314217 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:12.314104    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k podName:6f6d714d-426c-4f54-9c6b-64bd3bc8cd13 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:44.314084212 +0000 UTC m=+66.266510752 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-9996k" (UniqueName: "kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k") pod "network-check-target-kbtzf" (UID: "6f6d714d-426c-4f54-9c6b-64bd3bc8cd13") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 01 17:16:13.542665 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.542625    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:13.542665 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.542665    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:13.543197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.542745    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:13.543197 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:13.542765    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kbtzf" podUID="6f6d714d-426c-4f54-9c6b-64bd3bc8cd13"
Jul 01 17:16:13.543197 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:13.542855    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wqwfq" podUID="0496a8d7-d920-4aea-987a-1903d992115b"
Jul 01 17:16:13.543197 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:13.542963    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:16:13.809115 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.809037    2585 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-93.ec2.internal" event="NodeReady"
Jul 01 17:16:13.809268 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.809190    2585 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 01 17:16:13.854039 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.854002    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7xfvp"]
Jul 01 17:16:13.856911 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.856885    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:13.857055 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.856939    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rvvk6"]
Jul 01 17:16:13.858738 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.858542    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:13.859386 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.859365    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 01 17:16:13.859487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.859404    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pcm4c\""
Jul 01 17:16:13.859487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.859403    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 01 17:16:13.860995 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.860975    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5zq94\""
Jul 01 17:16:13.861106 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.861010    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 01 17:16:13.861106 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.861019    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 01 17:16:13.861258 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.861234    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 01 17:16:13.869012 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.868856    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xfvp"]
Jul 01 17:16:13.870085 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.869430    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rvvk6"]
Jul 01 17:16:13.927447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927414    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:13.927447 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927452    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdpdw\" (UniqueName: \"kubernetes.io/projected/d86eb336-78a9-4647-9a6f-e426ca13af76-kube-api-access-zdpdw\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:13.927705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927485    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d86eb336-78a9-4647-9a6f-e426ca13af76-config-volume\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:13.927705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927628    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86eb336-78a9-4647-9a6f-e426ca13af76-tmp-dir\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:13.927705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927667    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-cert\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:13.927705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:13.927693    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxn6v\" (UniqueName: \"kubernetes.io/projected/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-kube-api-access-pxn6v\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.028931 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.028883    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.028931 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.028925    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdpdw\" (UniqueName: \"kubernetes.io/projected/d86eb336-78a9-4647-9a6f-e426ca13af76-kube-api-access-zdpdw\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.028952    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d86eb336-78a9-4647-9a6f-e426ca13af76-config-volume\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.028989    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86eb336-78a9-4647-9a6f-e426ca13af76-tmp-dir\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.029014    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-cert\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.029036    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxn6v\" (UniqueName: \"kubernetes.io/projected/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-kube-api-access-pxn6v\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:14.029055    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:14.029190 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:14.029126    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:14.529104982 +0000 UTC m=+36.481531743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:14.029484 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.029382    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d86eb336-78a9-4647-9a6f-e426ca13af76-tmp-dir\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.029752 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.029731    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d86eb336-78a9-4647-9a6f-e426ca13af76-config-volume\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.034380 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.034351    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-cert\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.039855 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.037940    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxn6v\" (UniqueName: \"kubernetes.io/projected/d8ecef00-bbd7-4c6b-a5bb-53d8c914679c-kube-api-access-pxn6v\") pod \"ingress-canary-rvvk6\" (UID: \"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c\") " pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.040449 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.040418    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdpdw\" (UniqueName: \"kubernetes.io/projected/d86eb336-78a9-4647-9a6f-e426ca13af76-kube-api-access-zdpdw\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.177723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.177575    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rvvk6"
Jul 01 17:16:14.533674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.533635    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:14.533862 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:14.533801    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:14.533918 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:14.533866    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:15.533849949 +0000 UTC m=+37.486276474 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:14.969553 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:14.969521    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rvvk6"]
Jul 01 17:16:14.973293 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:16:14.973258    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ecef00_bbd7_4c6b_a5bb_53d8c914679c.slice/crio-2ad1cf2dc78227b130e1cb7df82eba666166660d7a5312a2c74a10a0df1288f5 WatchSource:0}: Error finding container 2ad1cf2dc78227b130e1cb7df82eba666166660d7a5312a2c74a10a0df1288f5: Status 404 returned error can't find the container with id 2ad1cf2dc78227b130e1cb7df82eba666166660d7a5312a2c74a10a0df1288f5
Jul 01 17:16:15.542056 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.542012    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:15.542056 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.542049    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:15.542327 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.542060    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:15.542327 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.542060    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:15.542327 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:15.542259    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:15.542327 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:15.542323    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:17.54230478 +0000 UTC m=+39.494731319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:15.545771 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545742    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-875gr\""
Jul 01 17:16:15.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545802    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 17:16:15.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545823    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 17:16:15.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545853    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 17:16:15.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545800    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 17:16:15.545904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.545753    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnmdh\""
Jul 01 17:16:15.708901 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.708862    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rvvk6" event={"ID":"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c","Type":"ContainerStarted","Data":"2ad1cf2dc78227b130e1cb7df82eba666166660d7a5312a2c74a10a0df1288f5"}
Jul 01 17:16:15.711579 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.711548    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="7835dfa4dbd6d52992d9e23a968fff4857433751cd8b48dfa231a228f15808a1" exitCode=0
Jul 01 17:16:15.711717 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:15.711614    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"7835dfa4dbd6d52992d9e23a968fff4857433751cd8b48dfa231a228f15808a1"}
Jul 01 17:16:16.718408 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:16.718162    2585 generic.go:356] "Generic (PLEG): container finished" podID="b1f7031e-8809-41a2-b19d-3f403e8e1942" containerID="872dee640b2cf7b3a9c9ba93f48e083bc262f40d2b57b7ad804cf742072bde38" exitCode=0
Jul 01 17:16:16.718875 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:16.718241    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerDied","Data":"872dee640b2cf7b3a9c9ba93f48e083bc262f40d2b57b7ad804cf742072bde38"}
Jul 01 17:16:17.556935 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:17.556831    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:17.557109 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:17.556961    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:17.557109 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:17.557019    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:21.557002838 +0000 UTC m=+43.509429379 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:17.725189 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:17.725146    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" event={"ID":"b1f7031e-8809-41a2-b19d-3f403e8e1942","Type":"ContainerStarted","Data":"2dc171f5de1145ce05daa2b249ec22b8daf9249925b234b8dd4a0fb259cfe23e"}
Jul 01 17:16:17.749849 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:17.749786    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7tvs4" podStartSLOduration=8.515685133 podStartE2EDuration="39.749764538s" totalImagesPullingTime="31.234079405s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:17.748422168 +0000 UTC m=+39.700848718" watchObservedRunningTime="2026-07-01 17:16:17.749764538 +0000 UTC m=+39.702191087"
Jul 01 17:16:19.732133 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:19.732090    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rvvk6" event={"ID":"d8ecef00-bbd7-4c6b-a5bb-53d8c914679c","Type":"ContainerStarted","Data":"d6cbf68af9ee989fbbb0dcc91b9e6ef109ae6c7efcd2c7ba67750d4f9bd70d0d"}
Jul 01 17:16:19.749556 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:19.749506    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rvvk6" podStartSLOduration=2.946136164 podStartE2EDuration="6.749489259s" totalImagesPullingTime="3.803353095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:19.748874868 +0000 UTC m=+41.701301414" watchObservedRunningTime="2026-07-01 17:16:19.749489259 +0000 UTC m=+41.701915804"
Jul 01 17:16:21.585313 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:21.585272    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:21.585749 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:21.585442    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:21.585749 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:21.585523    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:29.585507468 +0000 UTC m=+51.537933992 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:29.642965 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:29.642928    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:29.643453 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:29.643088    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:29.643453 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:29.643155    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:16:45.643137084 +0000 UTC m=+67.595563608 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:39.699135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:39.699105    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qgf8j"
Jul 01 17:16:44.246979 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.246940    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:16:44.246979 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.246989    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:44.249540 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.249513    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 01 17:16:44.249661 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.249620    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 01 17:16:44.257653 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:44.257633    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 17:16:44.257739 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:44.257703    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:17:48.257680307 +0000 UTC m=+130.210106849 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : secret "metrics-daemon-secret" not found
Jul 01 17:16:44.260907 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.260888    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0496a8d7-d920-4aea-987a-1903d992115b-original-pull-secret\") pod \"global-pull-secret-syncer-wqwfq\" (UID: \"0496a8d7-d920-4aea-987a-1903d992115b\") " pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:44.347304 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.347264    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:44.349812 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.349794    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 01 17:16:44.351803 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.351790    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wqwfq"
Jul 01 17:16:44.360551 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.360526    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 01 17:16:44.371084 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.371054    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9996k\" (UniqueName: \"kubernetes.io/projected/6f6d714d-426c-4f54-9c6b-64bd3bc8cd13-kube-api-access-9996k\") pod \"network-check-target-kbtzf\" (UID: \"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13\") " pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:44.475635 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.475606    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wqwfq"]
Jul 01 17:16:44.478965 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:16:44.478933    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0496a8d7_d920_4aea_987a_1903d992115b.slice/crio-7d58d86f852952edc72e05d113f11a180c92ef6081eda41dfec9a14b3119448e WatchSource:0}: Error finding container 7d58d86f852952edc72e05d113f11a180c92ef6081eda41dfec9a14b3119448e: Status 404 returned error can't find the container with id 7d58d86f852952edc72e05d113f11a180c92ef6081eda41dfec9a14b3119448e
Jul 01 17:16:44.659996 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.659907    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-875gr\""
Jul 01 17:16:44.668363 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.668333    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:44.791126 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.791097    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqwfq" event={"ID":"0496a8d7-d920-4aea-987a-1903d992115b","Type":"ContainerStarted","Data":"7d58d86f852952edc72e05d113f11a180c92ef6081eda41dfec9a14b3119448e"}
Jul 01 17:16:44.795390 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:44.795343    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kbtzf"]
Jul 01 17:16:44.797841 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:16:44.797812    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f6d714d_426c_4f54_9c6b_64bd3bc8cd13.slice/crio-e86f304c2f925b782304d4076a8ae1e3cb2f6681e7d68fe705d25b0cb7695dab WatchSource:0}: Error finding container e86f304c2f925b782304d4076a8ae1e3cb2f6681e7d68fe705d25b0cb7695dab: Status 404 returned error can't find the container with id e86f304c2f925b782304d4076a8ae1e3cb2f6681e7d68fe705d25b0cb7695dab
Jul 01 17:16:45.656395 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:45.656354    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:16:45.656923 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:45.656510    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:16:45.656923 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:16:45.656582    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:17:17.656561491 +0000 UTC m=+99.608988019 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:16:45.795316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:45.795277    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kbtzf" event={"ID":"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13","Type":"ContainerStarted","Data":"e86f304c2f925b782304d4076a8ae1e3cb2f6681e7d68fe705d25b0cb7695dab"}
Jul 01 17:16:47.801903 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:47.801869    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kbtzf" event={"ID":"6f6d714d-426c-4f54-9c6b-64bd3bc8cd13","Type":"ContainerStarted","Data":"a6eb69d9e9ac0ce1887c4f25bdfa2b3c118bb2b9c90c4f57fb21b75191087369"}
Jul 01 17:16:47.802337 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:47.802001    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:16:47.819248 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:47.819199    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kbtzf" podStartSLOduration=66.940651446 podStartE2EDuration="1m9.819184824s" totalImagesPullingTime="2.878533378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:47.818909538 +0000 UTC m=+69.771336084" watchObservedRunningTime="2026-07-01 17:16:47.819184824 +0000 UTC m=+69.771611381"
Jul 01 17:16:49.808911 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:49.808868    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wqwfq" event={"ID":"0496a8d7-d920-4aea-987a-1903d992115b","Type":"ContainerStarted","Data":"8cc7149d0fdc7b6efde67ed226000a3c248cfad9fd7d0a257d412a311926e94e"}
Jul 01 17:16:49.827962 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:16:49.827902    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wqwfq" podStartSLOduration=66.943903717 podStartE2EDuration="1m11.827880098s" totalImagesPullingTime="4.883976381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:16:49.827261366 +0000 UTC m=+71.779687912" watchObservedRunningTime="2026-07-01 17:16:49.827880098 +0000 UTC m=+71.780306646"
Jul 01 17:17:17.675659 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:17:17.675567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:17:17.676049 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:17:17.675748    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:17:17.676049 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:17:17.675831    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:18:21.675812596 +0000 UTC m=+163.628239120 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:17:18.807485 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:17:18.807459    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kbtzf"
Jul 01 17:17:48.287269 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:17:48.287212    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:17:48.287804 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:17:48.287376    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 17:17:48.287804 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:17:48.287456    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:50.28743717 +0000 UTC m=+252.239863693 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : secret "metrics-daemon-secret" not found
Jul 01 17:18:16.870875 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:18:16.870820    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7xfvp" podUID="d86eb336-78a9-4647-9a6f-e426ca13af76"
Jul 01 17:18:17.016832 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:18:17.016797    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:18:18.561844 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:18:18.561806    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:18:21.719571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:18:21.719523    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:18:21.719973 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:18:21.719662    2585 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 01 17:18:21.719973 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:18:21.719726    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls podName:d86eb336-78a9-4647-9a6f-e426ca13af76 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:23.719710392 +0000 UTC m=+285.672136916 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls") pod "dns-default-7xfvp" (UID: "d86eb336-78a9-4647-9a6f-e426ca13af76") : secret "dns-default-metrics-tls" not found
Jul 01 17:18:32.542378 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:18:32.542289    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:19:38.064222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.064171    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-4md7h"]
Jul 01 17:19:38.066181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.066164    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h"
Jul 01 17:19:38.073143 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.073120    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-89pvx\""
Jul 01 17:19:38.087198 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.087167    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-4md7h"]
Jul 01 17:19:38.116712 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.116680    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cs8j\" (UniqueName: \"kubernetes.io/projected/b640d14e-3b23-4cd3-9eb1-e8bad24ac69c-kube-api-access-9cs8j\") pod \"network-check-source-56499946bc-4md7h\" (UID: \"b640d14e-3b23-4cd3-9eb1-e8bad24ac69c\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h"
Jul 01 17:19:38.149953 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.149916    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"]
Jul 01 17:19:38.151934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.151919    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"
Jul 01 17:19:38.154308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.154281    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 01 17:19:38.154308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.154302    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 01 17:19:38.154497 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.154340    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ggrtg\""
Jul 01 17:19:38.158611 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.158489    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6fd9c6d85c-8vt25"]
Jul 01 17:19:38.160652 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.160633    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"]
Jul 01 17:19:38.160789 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.160769    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.162670 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.162653    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"]
Jul 01 17:19:38.162813 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.162797    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.164128 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164106    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 01 17:19:38.164636 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164615    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"]
Jul 01 17:19:38.164754 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164736    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.164916 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164835    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 01 17:19:38.164916 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164894    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 01 17:19:38.165040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.164841    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kqqxh\""
Jul 01 17:19:38.165103 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165061    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 01 17:19:38.165156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165114    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 01 17:19:38.165207 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165174    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 01 17:19:38.165415 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165395    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 01 17:19:38.165917 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165898    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9c2rc\""
Jul 01 17:19:38.166044 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165923    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 01 17:19:38.166154 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.166061    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 01 17:19:38.166154 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.165977    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 01 17:19:38.167110 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.167081    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 01 17:19:38.167110 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.167082    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fxmmk\""
Jul 01 17:19:38.167635 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.167431    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 01 17:19:38.168615 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.168574    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 01 17:19:38.168615 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.168608    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 01 17:19:38.172146 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.172129    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 01 17:19:38.176558 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.176539    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"]
Jul 01 17:19:38.177285 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.177263    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fd9c6d85c-8vt25"]
Jul 01 17:19:38.192207 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.192181    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"]
Jul 01 17:19:38.217795 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217758    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cs8j\" (UniqueName: \"kubernetes.io/projected/b640d14e-3b23-4cd3-9eb1-e8bad24ac69c-kube-api-access-9cs8j\") pod \"network-check-source-56499946bc-4md7h\" (UID: \"b640d14e-3b23-4cd3-9eb1-e8bad24ac69c\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h"
Jul 01 17:19:38.217795 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217799    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rvs7\" (UniqueName: \"kubernetes.io/projected/f26a7708-bf17-4abb-8b42-d6c3c221b6cd-kube-api-access-6rvs7\") pod \"volume-data-source-validator-69d9f679db-rl8t4\" (UID: \"f26a7708-bf17-4abb-8b42-d6c3c221b6cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"
Jul 01 17:19:38.218001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217820    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-stats-auth\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.218001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217839    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.218001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217917    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f167518-49f7-43b0-909b-c78a72de0220-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.218001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217951    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pln2j\" (UniqueName: \"kubernetes.io/projected/8f167518-49f7-43b0-909b-c78a72de0220-kube-api-access-pln2j\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.218001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.217978    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbrwr\" (UniqueName: \"kubernetes.io/projected/1db61671-de63-445c-a5c0-ea12692510d0-kube-api-access-sbrwr\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.218153 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218048    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-config\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.218153 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218104    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1db61671-de63-445c-a5c0-ea12692510d0-serving-cert\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.218153 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218129    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.218153 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218146    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cvj8\" (UniqueName: \"kubernetes.io/projected/addc8041-53ea-41ac-9979-e5872297f067-kube-api-access-8cvj8\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.218266 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-default-certificate\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.218266 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218212    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f167518-49f7-43b0-909b-c78a72de0220-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.218266 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.218259    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.225466 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.225436    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cs8j\" (UniqueName: \"kubernetes.io/projected/b640d14e-3b23-4cd3-9eb1-e8bad24ac69c-kube-api-access-9cs8j\") pod \"network-check-source-56499946bc-4md7h\" (UID: \"b640d14e-3b23-4cd3-9eb1-e8bad24ac69c\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h"
Jul 01 17:19:38.319095 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319007    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f167518-49f7-43b0-909b-c78a72de0220-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.319095 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319047    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.319095 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319089    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rvs7\" (UniqueName: \"kubernetes.io/projected/f26a7708-bf17-4abb-8b42-d6c3c221b6cd-kube-api-access-6rvs7\") pod \"volume-data-source-validator-69d9f679db-rl8t4\" (UID: \"f26a7708-bf17-4abb-8b42-d6c3c221b6cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319106    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-stats-auth\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319120    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f167518-49f7-43b0-909b-c78a72de0220-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319154    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pln2j\" (UniqueName: \"kubernetes.io/projected/8f167518-49f7-43b0-909b-c78a72de0220-kube-api-access-pln2j\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319170    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbrwr\" (UniqueName: \"kubernetes.io/projected/1db61671-de63-445c-a5c0-ea12692510d0-kube-api-access-sbrwr\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.319368 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.319310    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:38.819287583 +0000 UTC m=+240.771714122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.319413    2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319474    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-config\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.319497    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:38.819471184 +0000 UTC m=+240.771897715 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : secret "router-metrics-certs-default" not found
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319535    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1db61671-de63-445c-a5c0-ea12692510d0-serving-cert\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319565    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8cvj8\" (UniqueName: \"kubernetes.io/projected/addc8041-53ea-41ac-9979-e5872297f067-kube-api-access-8cvj8\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.319705 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319644    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-default-certificate\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.320047 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.319730    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f167518-49f7-43b0-909b-c78a72de0220-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.320267 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.320239    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-config\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.320560 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.320535    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1db61671-de63-445c-a5c0-ea12692510d0-trusted-ca\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.321740 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.321716    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f167518-49f7-43b0-909b-c78a72de0220-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.321839 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.321796    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-stats-auth\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.321958 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.321938    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-default-certificate\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.321994 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.321974    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1db61671-de63-445c-a5c0-ea12692510d0-serving-cert\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.330870 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.330841    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rvs7\" (UniqueName: \"kubernetes.io/projected/f26a7708-bf17-4abb-8b42-d6c3c221b6cd-kube-api-access-6rvs7\") pod \"volume-data-source-validator-69d9f679db-rl8t4\" (UID: \"f26a7708-bf17-4abb-8b42-d6c3c221b6cd\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"
Jul 01 17:19:38.330972 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.330929    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cvj8\" (UniqueName: \"kubernetes.io/projected/addc8041-53ea-41ac-9979-e5872297f067-kube-api-access-8cvj8\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.331100 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.331079    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbrwr\" (UniqueName: \"kubernetes.io/projected/1db61671-de63-445c-a5c0-ea12692510d0-kube-api-access-sbrwr\") pod \"console-operator-7fdbdf9c99-q6xtn\" (UID: \"1db61671-de63-445c-a5c0-ea12692510d0\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.331237 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.331224    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pln2j\" (UniqueName: \"kubernetes.io/projected/8f167518-49f7-43b0-909b-c78a72de0220-kube-api-access-pln2j\") pod \"kube-storage-version-migrator-operator-79964ff6f9-q6sgz\" (UID: \"8f167518-49f7-43b0-909b-c78a72de0220\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.375291 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.375264    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h"
Jul 01 17:19:38.464207 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.464179    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ggrtg\""
Jul 01 17:19:38.471981 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.471954    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"
Jul 01 17:19:38.481063 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.480919    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9c2rc\""
Jul 01 17:19:38.486930 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.486907    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-fxmmk\""
Jul 01 17:19:38.488801 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.488780    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:38.495629 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.495609    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"
Jul 01 17:19:38.497685 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.497658    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-4md7h"]
Jul 01 17:19:38.503416 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:19:38.503391    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb640d14e_3b23_4cd3_9eb1_e8bad24ac69c.slice/crio-da0c767a2ab367a2dadd9b24430ad6a3045892b79e5c7f1c5c9864ad0b643d9e WatchSource:0}: Error finding container da0c767a2ab367a2dadd9b24430ad6a3045892b79e5c7f1c5c9864ad0b643d9e: Status 404 returned error can't find the container with id da0c767a2ab367a2dadd9b24430ad6a3045892b79e5c7f1c5c9864ad0b643d9e
Jul 01 17:19:38.621183 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.621107    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4"]
Jul 01 17:19:38.633011 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:19:38.632986    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26a7708_bf17_4abb_8b42_d6c3c221b6cd.slice/crio-2a1ffa09df8f4f1785c785f1d94d1c32af0928b4879268ae06f39238ebde9af1 WatchSource:0}: Error finding container 2a1ffa09df8f4f1785c785f1d94d1c32af0928b4879268ae06f39238ebde9af1: Status 404 returned error can't find the container with id 2a1ffa09df8f4f1785c785f1d94d1c32af0928b4879268ae06f39238ebde9af1
Jul 01 17:19:38.641328 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.641249    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"]
Jul 01 17:19:38.644708 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:19:38.644682    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db61671_de63_445c_a5c0_ea12692510d0.slice/crio-840d70a18f3f92cf48d6211a2122d95ed4efb7f13b8f7614a5aff60f164550ca WatchSource:0}: Error finding container 840d70a18f3f92cf48d6211a2122d95ed4efb7f13b8f7614a5aff60f164550ca: Status 404 returned error can't find the container with id 840d70a18f3f92cf48d6211a2122d95ed4efb7f13b8f7614a5aff60f164550ca
Jul 01 17:19:38.658010 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.657976    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz"]
Jul 01 17:19:38.660619 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:19:38.660581    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f167518_49f7_43b0_909b_c78a72de0220.slice/crio-32b33fca4d6a0c766c6f0a2dfa3adb40f7037afecb2fb0467ab8dfa104de055a WatchSource:0}: Error finding container 32b33fca4d6a0c766c6f0a2dfa3adb40f7037afecb2fb0467ab8dfa104de055a: Status 404 returned error can't find the container with id 32b33fca4d6a0c766c6f0a2dfa3adb40f7037afecb2fb0467ab8dfa104de055a
Jul 01 17:19:38.824945 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.824870    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.824945 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:38.824923    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:38.825102 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.825001    2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 17:19:38.825102 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.825052    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:39.825036129 +0000 UTC m=+241.777462653 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : secret "router-metrics-certs-default" not found
Jul 01 17:19:38.825102 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:38.825069    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:39.82506319 +0000 UTC m=+241.777489714 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:39.207441 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.207347    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4" event={"ID":"f26a7708-bf17-4abb-8b42-d6c3c221b6cd","Type":"ContainerStarted","Data":"2a1ffa09df8f4f1785c785f1d94d1c32af0928b4879268ae06f39238ebde9af1"}
Jul 01 17:19:39.208720 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.208685    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" event={"ID":"1db61671-de63-445c-a5c0-ea12692510d0","Type":"ContainerStarted","Data":"840d70a18f3f92cf48d6211a2122d95ed4efb7f13b8f7614a5aff60f164550ca"}
Jul 01 17:19:39.210347 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.210310    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h" event={"ID":"b640d14e-3b23-4cd3-9eb1-e8bad24ac69c","Type":"ContainerStarted","Data":"552b030bd9f95d7e5c3417d27cc4f349034569ae9b7cd3368eb28de70e62d447"}
Jul 01 17:19:39.210451 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.210357    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h" event={"ID":"b640d14e-3b23-4cd3-9eb1-e8bad24ac69c","Type":"ContainerStarted","Data":"da0c767a2ab367a2dadd9b24430ad6a3045892b79e5c7f1c5c9864ad0b643d9e"}
Jul 01 17:19:39.211854 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.211830    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz" event={"ID":"8f167518-49f7-43b0-909b-c78a72de0220","Type":"ContainerStarted","Data":"32b33fca4d6a0c766c6f0a2dfa3adb40f7037afecb2fb0467ab8dfa104de055a"}
Jul 01 17:19:39.225414 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.225362    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-4md7h" podStartSLOduration=1.22534384 podStartE2EDuration="1.22534384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:19:39.224717921 +0000 UTC m=+241.177144469" watchObservedRunningTime="2026-07-01 17:19:39.22534384 +0000 UTC m=+241.177770386"
Jul 01 17:19:39.834410 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.834371    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:39.834625 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:39.834465    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:39.834625 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:39.834606    2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 17:19:39.834751 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:39.834668    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:41.834649619 +0000 UTC m=+243.787076149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : secret "router-metrics-certs-default" not found
Jul 01 17:19:39.835335 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:39.835079    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:41.835060503 +0000 UTC m=+243.787487032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:40.216966 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:40.216878    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4" event={"ID":"f26a7708-bf17-4abb-8b42-d6c3c221b6cd","Type":"ContainerStarted","Data":"4db224bc2ebe2abff980c8461b2c313d4f7ff98655e19372231f1b3759fc235c"}
Jul 01 17:19:40.234292 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:40.234242    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-rl8t4" podStartSLOduration=0.881865008 podStartE2EDuration="2.234228376s" totalImagesPullingTime="1.352363368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:19:40.233505724 +0000 UTC m=+242.185932273" watchObservedRunningTime="2026-07-01 17:19:40.234228376 +0000 UTC m=+242.186654921"
Jul 01 17:19:41.221879 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:41.221797    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz" event={"ID":"8f167518-49f7-43b0-909b-c78a72de0220","Type":"ContainerStarted","Data":"72e9ea845848df9d703b08fe430fed10850387d971137f7ac4e37f1971fe1341"}
Jul 01 17:19:41.237672 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:41.237612    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz" podStartSLOduration=1.074216618 podStartE2EDuration="3.237578345s" totalImagesPullingTime="2.163361727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:19:41.236966801 +0000 UTC m=+243.189393349" watchObservedRunningTime="2026-07-01 17:19:41.237578345 +0000 UTC m=+243.190004934"
Jul 01 17:19:41.852830 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:41.852746    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:41.852830 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:41.852807    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:41.853038 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:41.852920    2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 17:19:41.853038 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:41.852929    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:45.852914 +0000 UTC m=+247.805340523 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:41.853038 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:41.852994    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:45.852976299 +0000 UTC m=+247.805402829 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : secret "router-metrics-certs-default" not found
Jul 01 17:19:42.164345 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.164263    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qknjv"]
Jul 01 17:19:42.166531 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.166510    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.170637 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.170615    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 01 17:19:42.170875 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.170858    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 01 17:19:42.170953 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.170886    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 01 17:19:42.171127 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.171113    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 01 17:19:42.171947 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.171931    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-b7kxs\""
Jul 01 17:19:42.185112 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.185090    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qknjv"]
Jul 01 17:19:42.226389 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.226370    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/0.log"
Jul 01 17:19:42.226709 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.226404    2585 generic.go:356] "Generic (PLEG): container finished" podID="1db61671-de63-445c-a5c0-ea12692510d0" containerID="80c6594ade0bf5810a00b18a88e1abb2ad2481d1fb06bb57a8599cb1e90ba387" exitCode=255
Jul 01 17:19:42.226709 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.226475    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" event={"ID":"1db61671-de63-445c-a5c0-ea12692510d0","Type":"ContainerDied","Data":"80c6594ade0bf5810a00b18a88e1abb2ad2481d1fb06bb57a8599cb1e90ba387"}
Jul 01 17:19:42.226777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.226730    2585 scope.go:117] "RemoveContainer" containerID="80c6594ade0bf5810a00b18a88e1abb2ad2481d1fb06bb57a8599cb1e90ba387"
Jul 01 17:19:42.255971 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.255949    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-crio-socket\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.256069 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.255990    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmlvg\" (UniqueName: \"kubernetes.io/projected/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-api-access-hmlvg\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.256069 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.256062    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.256135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.256084    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-data-volume\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.256135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.256110    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.356960 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.356934    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357093 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.357042    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:42.357149 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.357100    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:42.857080542 +0000 UTC m=+244.809507085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:42.357149 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357095    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-data-volume\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357249 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357159    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357249 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357213    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-crio-socket\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357324 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357264    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmlvg\" (UniqueName: \"kubernetes.io/projected/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-api-access-hmlvg\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357324 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.357290    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 17:19:42.357403 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.357341    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:42.857328254 +0000 UTC m=+244.809754793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : configmap "kube-rbac-proxy" not found
Jul 01 17:19:42.357403 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357373    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-crio-socket\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.357403 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.357373    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-data-volume\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.364978 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.364958    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmlvg\" (UniqueName: \"kubernetes.io/projected/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-api-access-hmlvg\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.861421 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.861375    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.861577 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:42.861479    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:42.861577 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.861492    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 17:19:42.861577 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.861548    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:43.861531885 +0000 UTC m=+245.813958413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : configmap "kube-rbac-proxy" not found
Jul 01 17:19:42.861577 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.861572    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:42.861775 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:42.861628    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:43.861616497 +0000 UTC m=+245.814043024 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:43.231393 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.231323    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/1.log"
Jul 01 17:19:43.231954 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.231757    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/0.log"
Jul 01 17:19:43.231954 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.231797    2585 generic.go:356] "Generic (PLEG): container finished" podID="1db61671-de63-445c-a5c0-ea12692510d0" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54" exitCode=255
Jul 01 17:19:43.231954 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.231835    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" event={"ID":"1db61671-de63-445c-a5c0-ea12692510d0","Type":"ContainerDied","Data":"f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"}
Jul 01 17:19:43.231954 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.231885    2585 scope.go:117] "RemoveContainer" containerID="80c6594ade0bf5810a00b18a88e1abb2ad2481d1fb06bb57a8599cb1e90ba387"
Jul 01 17:19:43.232138 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.232087    2585 scope.go:117] "RemoveContainer" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"
Jul 01 17:19:43.232297 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:43.232276    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:19:43.869166 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.869125    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:43.869358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:43.869185    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:43.869358 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:43.869293    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:43.869358 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:43.869308    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 17:19:43.869466 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:43.869368    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:45.869353784 +0000 UTC m=+247.821780308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:43.869466 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:43.869386    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:45.869379362 +0000 UTC m=+247.821805885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : configmap "kube-rbac-proxy" not found
Jul 01 17:19:44.164841 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:44.164772    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fgjrz_fe4d7fa6-f0ff-435d-98b8-80e993e6a414/dns-node-resolver/0.log"
Jul 01 17:19:44.235941 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:44.235918    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/1.log"
Jul 01 17:19:44.236321 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:44.236231    2585 scope.go:117] "RemoveContainer" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"
Jul 01 17:19:44.236390 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:44.236372    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:19:45.094808 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.094777    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"]
Jul 01 17:19:45.097313 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.097298    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.099892 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.099870    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 01 17:19:45.099892 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.099881    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-l7bwk\""
Jul 01 17:19:45.100861 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.100827    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 01 17:19:45.100916 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.100862    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 01 17:19:45.100959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.100862    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 01 17:19:45.106656 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.106638    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"]
Jul 01 17:19:45.164570 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.164549    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mw5ct_4832d76a-9f98-422b-adc6-e44703c5248f/node-ca/0.log"
Jul 01 17:19:45.180946 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.180922    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.181030 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.180976    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-key\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.181030 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.181020    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbsmh\" (UniqueName: \"kubernetes.io/projected/d84be6c6-d210-41b9-ba83-611829a7f72d-kube-api-access-pbsmh\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.281750 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.281721    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-key\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.282057 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.281755    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbsmh\" (UniqueName: \"kubernetes.io/projected/d84be6c6-d210-41b9-ba83-611829a7f72d-kube-api-access-pbsmh\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.282057 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.281993    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.282543 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.282526    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.284097 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.284080    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d84be6c6-d210-41b9-ba83-611829a7f72d-signing-key\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.290086 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.290065    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbsmh\" (UniqueName: \"kubernetes.io/projected/d84be6c6-d210-41b9-ba83-611829a7f72d-kube-api-access-pbsmh\") pod \"service-ca-7bb4f4bcc7-z5mdj\" (UID: \"d84be6c6-d210-41b9-ba83-611829a7f72d\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.406617 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.406499    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"
Jul 01 17:19:45.523137 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.523106    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj"]
Jul 01 17:19:45.525808 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:19:45.525777    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84be6c6_d210_41b9_ba83_611829a7f72d.slice/crio-5257db2880dbc6642b5c1a4742b8638b3b0aa2609968a0d5294faeb32ad8821f WatchSource:0}: Error finding container 5257db2880dbc6642b5c1a4742b8638b3b0aa2609968a0d5294faeb32ad8821f: Status 404 returned error can't find the container with id 5257db2880dbc6642b5c1a4742b8638b3b0aa2609968a0d5294faeb32ad8821f
Jul 01 17:19:45.886823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.886784    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:45.886823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.886829    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.886904    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.886939    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.886939    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:53.886919945 +0000 UTC m=+255.839346469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.886987    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:49.886975243 +0000 UTC m=+251.839401768 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : configmap "kube-rbac-proxy" not found
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.886992    2585 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 01 17:19:45.887019 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:45.887007    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:45.887206 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.887036    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:53.887023096 +0000 UTC m=+255.839449620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : secret "router-metrics-certs-default" not found
Jul 01 17:19:45.887206 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.887068    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:45.887206 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:45.887091    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:49.887083151 +0000 UTC m=+251.839509675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:46.166118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:46.166007    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rvvk6_d8ecef00-bbd7-4c6b-a5bb-53d8c914679c/serve-healthcheck-canary/0.log"
Jul 01 17:19:46.243605 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:46.243557    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj" event={"ID":"d84be6c6-d210-41b9-ba83-611829a7f72d","Type":"ContainerStarted","Data":"5257db2880dbc6642b5c1a4742b8638b3b0aa2609968a0d5294faeb32ad8821f"}
Jul 01 17:19:48.250463 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:48.250427    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj" event={"ID":"d84be6c6-d210-41b9-ba83-611829a7f72d","Type":"ContainerStarted","Data":"d9d15e237c181f7e904b1635241d1d335d38a9934404a5fadf5d9b01e3684eb8"}
Jul 01 17:19:48.267720 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:48.267675    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-z5mdj" podStartSLOduration=0.921736159 podStartE2EDuration="3.267659826s" totalImagesPullingTime="2.345923667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:19:48.2667147 +0000 UTC m=+250.219141271" watchObservedRunningTime="2026-07-01 17:19:48.267659826 +0000 UTC m=+250.220086371"
Jul 01 17:19:48.491486 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:48.490542    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:48.491486 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:48.490617    2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:19:48.491486 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:48.490987    2585 scope.go:117] "RemoveContainer" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"
Jul 01 17:19:48.491486 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:48.491152    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:19:49.922655 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:49.922589    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:49.922995 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:49.922694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:49.922995 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:49.922697    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:49.922995 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:49.922756    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:57.922741468 +0000 UTC m=+259.875167995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:49.922995 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:49.922777    2585 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 01 17:19:49.922995 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:49.922828    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:19:57.922813223 +0000 UTC m=+259.875239747 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : configmap "kube-rbac-proxy" not found
Jul 01 17:19:50.325212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:50.325174    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:19:50.325388 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:50.325336    2585 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 01 17:19:50.325429 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:50.325396    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs podName:ba2d162e-5b36-4378-a440-669ffe4fc9d4 nodeName:}" failed. No retries permitted until 2026-07-01 17:21:52.325378166 +0000 UTC m=+374.277804690 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs") pod "network-metrics-daemon-2724j" (UID: "ba2d162e-5b36-4378-a440-669ffe4fc9d4") : secret "metrics-daemon-secret" not found
Jul 01 17:19:53.951895 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:53.951861    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:53.952364 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:53.951918    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:53.952364 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:53.952061    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle podName:addc8041-53ea-41ac-9979-e5872297f067 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:09.952038895 +0000 UTC m=+271.904465430 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle") pod "router-default-6fd9c6d85c-8vt25" (UID: "addc8041-53ea-41ac-9979-e5872297f067") : configmap references non-existent config key: service-ca.crt
Jul 01 17:19:53.954282 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:53.954260    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/addc8041-53ea-41ac-9979-e5872297f067-metrics-certs\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:19:57.984142 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:57.984098    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:57.984655 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:57.984179    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:19:57.984655 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:57.984358    2585 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:57.984655 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:19:57.984444    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls podName:76fa0efc-dc7e-41d3-b077-1ba2afa70be1 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:13.984420044 +0000 UTC m=+275.936846572 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qknjv" (UID: "76fa0efc-dc7e-41d3-b077-1ba2afa70be1") : secret "insights-runtime-extractor-tls" not found
Jul 01 17:19:57.984854 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:19:57.984690    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:20:00.542562 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:00.542472    2585 scope.go:117] "RemoveContainer" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"
Jul 01 17:20:01.289982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.289950    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:20:01.290357 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.290339    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/1.log"
Jul 01 17:20:01.290411 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.290375    2585 generic.go:356] "Generic (PLEG): container finished" podID="1db61671-de63-445c-a5c0-ea12692510d0" containerID="276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60" exitCode=255
Jul 01 17:20:01.290448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.290421    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" event={"ID":"1db61671-de63-445c-a5c0-ea12692510d0","Type":"ContainerDied","Data":"276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60"}
Jul 01 17:20:01.290497 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.290465    2585 scope.go:117] "RemoveContainer" containerID="f7485da9e8650c7f74ff359a1dc92babd73bf131c825b409f5e8a6b16fd5ba54"
Jul 01 17:20:01.290866 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:01.290845    2585 scope.go:117] "RemoveContainer" containerID="276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60"
Jul 01 17:20:01.291071 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:01.291040    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:20:02.295282 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:02.295255    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:20:06.909580 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.909543    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86d6c8484-vxf2f"]
Jul 01 17:20:06.912699 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.912677    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:06.916312 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.916289    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 01 17:20:06.916448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.916395    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 01 17:20:06.917438 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.917421    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jkmfz\""
Jul 01 17:20:06.917637 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.917619    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 01 17:20:06.922285 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.922267    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 01 17:20:06.929111 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:06.929090    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d6c8484-vxf2f"]
Jul 01 17:20:07.052262 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052229    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-694hp\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-kube-api-access-694hp\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052269    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-trusted-ca\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052304    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3236e4b5-775e-46b8-ae63-752fb67a26c6-ca-trust-extracted\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-bound-sa-token\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052401    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-tls\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052421    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-certificates\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052437    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-installation-pull-secrets\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.052544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.052486    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-image-registry-private-configuration\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153046 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153017    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-694hp\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-kube-api-access-694hp\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153052    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-trusted-ca\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153083    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3236e4b5-775e-46b8-ae63-752fb67a26c6-ca-trust-extracted\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153124    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-bound-sa-token\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153151    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-tls\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153182    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-certificates\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153439 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153274    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-installation-pull-secrets\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153439 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153344    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-image-registry-private-configuration\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.153606 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.153572    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3236e4b5-775e-46b8-ae63-752fb67a26c6-ca-trust-extracted\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.154096 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.154070    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-certificates\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.154250 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.154233    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3236e4b5-775e-46b8-ae63-752fb67a26c6-trusted-ca\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.155825 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.155807    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-installation-pull-secrets\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.155899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.155864    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-registry-tls\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.155934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.155871    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3236e4b5-775e-46b8-ae63-752fb67a26c6-image-registry-private-configuration\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.162241 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.162189    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-694hp\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-kube-api-access-694hp\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.162241 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.162229    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3236e4b5-775e-46b8-ae63-752fb67a26c6-bound-sa-token\") pod \"image-registry-86d6c8484-vxf2f\" (UID: \"3236e4b5-775e-46b8-ae63-752fb67a26c6\") " pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.223211 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.223183    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:07.350631 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:07.350606    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86d6c8484-vxf2f"]
Jul 01 17:20:07.352749 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:07.352723    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3236e4b5_775e_46b8_ae63_752fb67a26c6.slice/crio-ecb8fdfa7b67306e4dbe06304ca7667fbef7dd289b294106a6329323baedf982 WatchSource:0}: Error finding container ecb8fdfa7b67306e4dbe06304ca7667fbef7dd289b294106a6329323baedf982: Status 404 returned error can't find the container with id ecb8fdfa7b67306e4dbe06304ca7667fbef7dd289b294106a6329323baedf982
Jul 01 17:20:08.314303 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.314268    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f" event={"ID":"3236e4b5-775e-46b8-ae63-752fb67a26c6","Type":"ContainerStarted","Data":"5c725b02149703d8fc5b473a2f152cc8a65386ebf1c2d939a115275c49bb424f"}
Jul 01 17:20:08.314721 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.314317    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f" event={"ID":"3236e4b5-775e-46b8-ae63-752fb67a26c6","Type":"ContainerStarted","Data":"ecb8fdfa7b67306e4dbe06304ca7667fbef7dd289b294106a6329323baedf982"}
Jul 01 17:20:08.314721 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.314377    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:08.334110 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.334065    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f" podStartSLOduration=2.334050667 podStartE2EDuration="2.334050667s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:08.33239443 +0000 UTC m=+270.284820976" watchObservedRunningTime="2026-07-01 17:20:08.334050667 +0000 UTC m=+270.286477213"
Jul 01 17:20:08.489900 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.489865    2585 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:20:08.490051 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.489918    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:20:08.490201 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:08.490182    2585 scope.go:117] "RemoveContainer" containerID="276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60"
Jul 01 17:20:08.490353 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:08.490330    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:20:09.975992 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:09.975949    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:09.976529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:09.976500    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/addc8041-53ea-41ac-9979-e5872297f067-service-ca-bundle\") pod \"router-default-6fd9c6d85c-8vt25\" (UID: \"addc8041-53ea-41ac-9979-e5872297f067\") " pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:10.275405 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:10.275376    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kqqxh\""
Jul 01 17:20:10.283509 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:10.283490    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:10.406934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:10.406908    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6fd9c6d85c-8vt25"]
Jul 01 17:20:10.409142 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:10.409117    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaddc8041_53ea_41ac_9979_e5872297f067.slice/crio-fc9c36f4f991a6f2e4e492c1adfdfd80fd010acb52c9c858786e6ba7dcbdce4f WatchSource:0}: Error finding container fc9c36f4f991a6f2e4e492c1adfdfd80fd010acb52c9c858786e6ba7dcbdce4f: Status 404 returned error can't find the container with id fc9c36f4f991a6f2e4e492c1adfdfd80fd010acb52c9c858786e6ba7dcbdce4f
Jul 01 17:20:11.325357 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:11.325324    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25" event={"ID":"addc8041-53ea-41ac-9979-e5872297f067","Type":"ContainerStarted","Data":"ed2659f20b0e07c3a1e16a029a4434571675d0883c881ba810e1bed3b4420051"}
Jul 01 17:20:11.325748 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:11.325384    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25" event={"ID":"addc8041-53ea-41ac-9979-e5872297f067","Type":"ContainerStarted","Data":"fc9c36f4f991a6f2e4e492c1adfdfd80fd010acb52c9c858786e6ba7dcbdce4f"}
Jul 01 17:20:11.345955 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:11.345908    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25" podStartSLOduration=33.345893826 podStartE2EDuration="33.345893826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:11.344878444 +0000 UTC m=+273.297304984" watchObservedRunningTime="2026-07-01 17:20:11.345893826 +0000 UTC m=+273.298320371"
Jul 01 17:20:12.284291 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:12.284252    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:12.286804 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:12.286786    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:12.328916 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:12.328884    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:12.330080 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:12.330061    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6fd9c6d85c-8vt25"
Jul 01 17:20:14.006085 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:14.006049    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:20:14.008391 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:14.008367    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/76fa0efc-dc7e-41d3-b077-1ba2afa70be1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qknjv\" (UID: \"76fa0efc-dc7e-41d3-b077-1ba2afa70be1\") " pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:20:14.275278 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:14.275176    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qknjv"
Jul 01 17:20:14.400662 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:14.400639    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qknjv"]
Jul 01 17:20:14.402954 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:14.402923    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76fa0efc_dc7e_41d3_b077_1ba2afa70be1.slice/crio-5d0709269e5995fbd87f9abf4f91d356267d2075f096d8394e4a129146665c0e WatchSource:0}: Error finding container 5d0709269e5995fbd87f9abf4f91d356267d2075f096d8394e4a129146665c0e: Status 404 returned error can't find the container with id 5d0709269e5995fbd87f9abf4f91d356267d2075f096d8394e4a129146665c0e
Jul 01 17:20:15.342355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:15.342312    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknjv" event={"ID":"76fa0efc-dc7e-41d3-b077-1ba2afa70be1","Type":"ContainerStarted","Data":"5d0709269e5995fbd87f9abf4f91d356267d2075f096d8394e4a129146665c0e"}
Jul 01 17:20:17.730675 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.730638    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-b8snh"]
Jul 01 17:20:17.733057 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.733036    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.736784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.736754    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 01 17:20:17.736901 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.736825    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 01 17:20:17.736966 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.736912    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 01 17:20:17.736966 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.736921    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-rhh6p\""
Jul 01 17:20:17.736966 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.736826    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 01 17:20:17.737233 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.737215    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 01 17:20:17.742333 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.742278    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-b8snh"]
Jul 01 17:20:17.836197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.836162    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcgsg\" (UniqueName: \"kubernetes.io/projected/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-kube-api-access-gcgsg\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.836388 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.836204    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.836388 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.836244    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.836388 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.836316    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.937246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.937203    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.937435 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.937313    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.937435 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.937382    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcgsg\" (UniqueName: \"kubernetes.io/projected/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-kube-api-access-gcgsg\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.937435 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.937412    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.938163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.938137    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.940279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.940255    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.940401 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.940375    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:17.949465 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:17.949429    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcgsg\" (UniqueName: \"kubernetes.io/projected/11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d-kube-api-access-gcgsg\") pod \"prometheus-operator-7b7d896497-b8snh\" (UID: \"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:18.044922 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:18.044902    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh"
Jul 01 17:20:18.186874 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:18.186849    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-b8snh"]
Jul 01 17:20:18.189146 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:18.189117    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11abdb0b_7b6c_4582_a903_2d3e2f6d0d6d.slice/crio-3c36547d5c629d6b67ba6a3bf767ffe925f8d71fc2b31a138ea7fc9372ff2f50 WatchSource:0}: Error finding container 3c36547d5c629d6b67ba6a3bf767ffe925f8d71fc2b31a138ea7fc9372ff2f50: Status 404 returned error can't find the container with id 3c36547d5c629d6b67ba6a3bf767ffe925f8d71fc2b31a138ea7fc9372ff2f50
Jul 01 17:20:18.354038 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:18.353956    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknjv" event={"ID":"76fa0efc-dc7e-41d3-b077-1ba2afa70be1","Type":"ContainerStarted","Data":"eeda653e87698ea0877891465348d45eb581c992f29e0d05e80b999fa7c9fcd8"}
Jul 01 17:20:18.355023 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:18.354994    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh" event={"ID":"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d","Type":"ContainerStarted","Data":"3c36547d5c629d6b67ba6a3bf767ffe925f8d71fc2b31a138ea7fc9372ff2f50"}
Jul 01 17:20:19.363489 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:19.363449    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknjv" event={"ID":"76fa0efc-dc7e-41d3-b077-1ba2afa70be1","Type":"ContainerStarted","Data":"57e03086441233a92e8915d6b534b056067fc4873b8f1f325062d9f0769ecb75"}
Jul 01 17:20:19.542125 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:19.542099    2585 scope.go:117] "RemoveContainer" containerID="276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60"
Jul 01 17:20:19.542313 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:19.542274    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-q6xtn_openshift-console-operator(1db61671-de63-445c-a5c0-ea12692510d0)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podUID="1db61671-de63-445c-a5c0-ea12692510d0"
Jul 01 17:20:20.018382 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:20.018336    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7xfvp" podUID="d86eb336-78a9-4647-9a6f-e426ca13af76"
Jul 01 17:20:20.373635 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:20.372294    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh" event={"ID":"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d","Type":"ContainerStarted","Data":"27c0b548b90e117841888a06b3c4122c607000303fcf085115279693bcbdc7e7"}
Jul 01 17:20:20.373635 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:20.372367    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh" event={"ID":"11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d","Type":"ContainerStarted","Data":"f59789a777acbc23557ad8135191efb9f04f386e091260b2714f31683aeb45e0"}
Jul 01 17:20:20.373635 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:20.372896    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:20.389031 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:20.388982    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-b8snh" podStartSLOduration=1.454584279 podStartE2EDuration="3.388969885s" totalImagesPullingTime="1.934385606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:20.388607096 +0000 UTC m=+282.341033637" watchObservedRunningTime="2026-07-01 17:20:20.388969885 +0000 UTC m=+282.341396430"
Jul 01 17:20:22.125054 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.125027    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-4sxzh"]
Jul 01 17:20:22.127612 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.127573    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.131617 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.131578    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 01 17:20:22.131753 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.131733    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 01 17:20:22.131753 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.131578    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 01 17:20:22.131988 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.131970    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-c9s7z\""
Jul 01 17:20:22.132919 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.132900    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qcc6m"]
Jul 01 17:20:22.135253 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.135235    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.138777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.138759    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 01 17:20:22.138867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.138852    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 01 17:20:22.139093 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.138992    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cd8st\""
Jul 01 17:20:22.139572 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.139557    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 01 17:20:22.141508 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.141487    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-4sxzh"]
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170659    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brshz\" (UniqueName: \"kubernetes.io/projected/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-kube-api-access-brshz\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170696    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170723    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170751    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-sys\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170778    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170807    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170863    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/312c257e-97f0-4c9f-b2d6-92f169d44db8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170887    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49c8b\" (UniqueName: \"kubernetes.io/projected/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-api-access-49c8b\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.170969 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.170931    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-wtmp\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171019    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-metrics-client-ca\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171056    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-root\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171080    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171103    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171159    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-textfile\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.171633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.171203    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272395 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272359    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272416    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brshz\" (UniqueName: \"kubernetes.io/projected/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-kube-api-access-brshz\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272441    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272456    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272476    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-sys\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272503    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272537    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/312c257e-97f0-4c9f-b2d6-92f169d44db8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272644    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49c8b\" (UniqueName: \"kubernetes.io/projected/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-api-access-49c8b\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272688    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-wtmp\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272722    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-metrics-client-ca\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272744    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-root\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272766    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272797    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.272905 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.272835    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-textfile\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.273294 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.273090    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-accelerators-collector-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.273294 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.273090    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-textfile\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.273693 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.273671    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-sys\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.273771 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.273738    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-metrics-client-ca\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.273897 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.273882    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-wtmp\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.274349 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.274322    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-metrics-client-ca\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.274432 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.274395    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-root\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.274518 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:22.274482    2585 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 01 17:20:22.274574 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:22.274533    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls podName:77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:22.774516181 +0000 UTC m=+284.726942713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls") pod "node-exporter-qcc6m" (UID: "77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9") : secret "node-exporter-tls" not found
Jul 01 17:20:22.274652 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.274571    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.274985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.274954    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/312c257e-97f0-4c9f-b2d6-92f169d44db8-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.275292 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.275273    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.276710 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.276672    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.282638 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.282615    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.288934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.288904    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49c8b\" (UniqueName: \"kubernetes.io/projected/312c257e-97f0-4c9f-b2d6-92f169d44db8-kube-api-access-49c8b\") pod \"kube-state-metrics-845b954455-4sxzh\" (UID: \"312c257e-97f0-4c9f-b2d6-92f169d44db8\") " pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.291749 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.291723    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brshz\" (UniqueName: \"kubernetes.io/projected/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-kube-api-access-brshz\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.382243 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.382164    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qknjv" event={"ID":"76fa0efc-dc7e-41d3-b077-1ba2afa70be1","Type":"ContainerStarted","Data":"16a14e5fa4ca07aaff450d7776d76a322057bf4d7826ba8c444725344c43d17a"}
Jul 01 17:20:22.404604 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.404547    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qknjv" podStartSLOduration=32.788838231 podStartE2EDuration="40.40452953s" totalImagesPullingTime="7.615691299s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:19:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:22.403844813 +0000 UTC m=+284.356271361" watchObservedRunningTime="2026-07-01 17:20:22.40452953 +0000 UTC m=+284.356956057"
Jul 01 17:20:22.457924 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.457895    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh"
Jul 01 17:20:22.630222 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:22.630181    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod312c257e_97f0_4c9f_b2d6_92f169d44db8.slice/crio-7f8483af259db57451a44e7e3e3e1f5b91c241f8a9cb9cf7493fec34cad5437e WatchSource:0}: Error finding container 7f8483af259db57451a44e7e3e3e1f5b91c241f8a9cb9cf7493fec34cad5437e: Status 404 returned error can't find the container with id 7f8483af259db57451a44e7e3e3e1f5b91c241f8a9cb9cf7493fec34cad5437e
Jul 01 17:20:22.640438 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.640412    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-4sxzh"]
Jul 01 17:20:22.777611 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.777567    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:22.779801 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:22.779782    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9-node-exporter-tls\") pod \"node-exporter-qcc6m\" (UID: \"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9\") " pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:23.075253 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.075221    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qcc6m"
Jul 01 17:20:23.087286 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:23.087252    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77d6bee6_8b03_4fbe_b0b8_5abc0b0182c9.slice/crio-8925e14bfd4e9e5ae04da244f7fa530bb832210486bbc272db095db88adf935a WatchSource:0}: Error finding container 8925e14bfd4e9e5ae04da244f7fa530bb832210486bbc272db095db88adf935a: Status 404 returned error can't find the container with id 8925e14bfd4e9e5ae04da244f7fa530bb832210486bbc272db095db88adf935a
Jul 01 17:20:23.271564 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.271531    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:20:23.277735 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.277704    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.281493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.281467    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 17:20:23.281643 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.281513    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 17:20:23.281643 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.281534    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l7bm9\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.283975    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284206    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284245    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284285    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284334    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284413    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 17:20:23.284504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.284492    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 17:20:23.299988 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.299907    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:20:23.382002 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.381923    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382002 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.381972    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382029    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382101    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382135    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382151    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382164    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382183    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382212    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382351    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjv4k\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382381    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.382643 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.382447    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.386248 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.386220    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcc6m" event={"ID":"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9","Type":"ContainerStarted","Data":"8925e14bfd4e9e5ae04da244f7fa530bb832210486bbc272db095db88adf935a"}
Jul 01 17:20:23.387442 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.387416    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh" event={"ID":"312c257e-97f0-4c9f-b2d6-92f169d44db8","Type":"ContainerStarted","Data":"7f8483af259db57451a44e7e3e3e1f5b91c241f8a9cb9cf7493fec34cad5437e"}
Jul 01 17:20:23.483383 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483353    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483383 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483389    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483418    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483445    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483493    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483544    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjv4k\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483570    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483606    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483644    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483694    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483736    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483799    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.483928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.483839    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.485484 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.484895    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.485484 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:23.484924    2585 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 01 17:20:23.485484 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:23.485009    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls podName:82d40e72-803e-4d35-8bb8-a6b5e0985b46 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:23.984987334 +0000 UTC m=+285.937413861 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46") : secret "alertmanager-main-tls" not found
Jul 01 17:20:23.485484 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.485219    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.485867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.485824    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.486828 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.486769    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.486828 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.486787    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.487726 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.487685    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.488177 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.488149    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.488177 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.488164    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.488294 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.488195    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.488696 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.488673    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.489117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.489100    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.494455 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.494412    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjv4k\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.785937 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.785897    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:23.788193 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.788167    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d86eb336-78a9-4647-9a6f-e426ca13af76-metrics-tls\") pod \"dns-default-7xfvp\" (UID: \"d86eb336-78a9-4647-9a6f-e426ca13af76\") " pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:23.976485 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.976449    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pcm4c\""
Jul 01 17:20:23.984750 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.984563    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:23.988103 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:23.988079    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:23.988520 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:23.988218    2585 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 01 17:20:23.988520 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:23.988283    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls podName:82d40e72-803e-4d35-8bb8-a6b5e0985b46 nodeName:}" failed. No retries permitted until 2026-07-01 17:20:24.988263704 +0000 UTC m=+286.940690231 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46") : secret "alertmanager-main-tls" not found
Jul 01 17:20:24.541443 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:24.541405    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7xfvp"]
Jul 01 17:20:24.546196 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:24.546149    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd86eb336_78a9_4647_9a6f_e426ca13af76.slice/crio-503147bcde64c32fd695fd2fbcb32541f4c22d7e5a4bf5ac0fbb08f2a159c04e WatchSource:0}: Error finding container 503147bcde64c32fd695fd2fbcb32541f4c22d7e5a4bf5ac0fbb08f2a159c04e: Status 404 returned error can't find the container with id 503147bcde64c32fd695fd2fbcb32541f4c22d7e5a4bf5ac0fbb08f2a159c04e
Jul 01 17:20:24.995693 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:24.995662    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:24.998282 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:24.998258    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:25.092112 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.092084    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:20:25.228562 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.228534    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:20:25.231052 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:25.231019    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82d40e72_803e_4d35_8bb8_a6b5e0985b46.slice/crio-65149ad03949dae32e45461e4ede56cca57fb44b79a7af20ed54269068a2f35f WatchSource:0}: Error finding container 65149ad03949dae32e45461e4ede56cca57fb44b79a7af20ed54269068a2f35f: Status 404 returned error can't find the container with id 65149ad03949dae32e45461e4ede56cca57fb44b79a7af20ed54269068a2f35f
Jul 01 17:20:25.397608 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.397509    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"65149ad03949dae32e45461e4ede56cca57fb44b79a7af20ed54269068a2f35f"}
Jul 01 17:20:25.398877 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.398846    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xfvp" event={"ID":"d86eb336-78a9-4647-9a6f-e426ca13af76","Type":"ContainerStarted","Data":"503147bcde64c32fd695fd2fbcb32541f4c22d7e5a4bf5ac0fbb08f2a159c04e"}
Jul 01 17:20:25.400646 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.400610    2585 generic.go:356] "Generic (PLEG): container finished" podID="77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9" containerID="3f90bff6ee283f5144fd35e2203f9eb54a00a00c986109b4ed6e89968928a3e8" exitCode=0
Jul 01 17:20:25.400975 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.400866    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcc6m" event={"ID":"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9","Type":"ContainerDied","Data":"3f90bff6ee283f5144fd35e2203f9eb54a00a00c986109b4ed6e89968928a3e8"}
Jul 01 17:20:25.404170 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.404146    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh" event={"ID":"312c257e-97f0-4c9f-b2d6-92f169d44db8","Type":"ContainerStarted","Data":"be999f5eef06d645d8261009d2aff21459f80f85c68380d29d7fd5106ac2a501"}
Jul 01 17:20:25.404293 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.404200    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh" event={"ID":"312c257e-97f0-4c9f-b2d6-92f169d44db8","Type":"ContainerStarted","Data":"a8b767ea52868c7ca2a5bfcaba0c55d4b8e5a1fd65276dcb673f98bea0775747"}
Jul 01 17:20:25.404293 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.404227    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh" event={"ID":"312c257e-97f0-4c9f-b2d6-92f169d44db8","Type":"ContainerStarted","Data":"9e9622c684293ba3b4fc4f4c083107953f58909ab11a0638adb90454904e54b9"}
Jul 01 17:20:25.437253 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:25.437201    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-4sxzh" podStartSLOduration=1.66250593 podStartE2EDuration="3.43718207s" totalImagesPullingTime="1.77467614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:25.435139334 +0000 UTC m=+287.387565882" watchObservedRunningTime="2026-07-01 17:20:25.43718207 +0000 UTC m=+287.389608617"
Jul 01 17:20:26.412225 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:26.412187    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcc6m" event={"ID":"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9","Type":"ContainerStarted","Data":"c6cff09fbb966ca20aeb35eff59cae20170fcba81a663450ffd7435c00f37b2c"}
Jul 01 17:20:26.412633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:26.412302    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qcc6m" event={"ID":"77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9","Type":"ContainerStarted","Data":"f293dd62ab7c0bd7d18039b730fc7cfed79d430811956423d315e845d764de03"}
Jul 01 17:20:26.433853 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:26.433806    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qcc6m" podStartSLOduration=3.023355222 podStartE2EDuration="4.433791575s" totalImagesPullingTime="1.319316266s" totalInitContainerRuntime="91.120087ms" isStatefulPod=true podCreationTimestamp="2026-07-01 17:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:26.432195722 +0000 UTC m=+288.384622268" watchObservedRunningTime="2026-07-01 17:20:26.433791575 +0000 UTC m=+288.386218197"
Jul 01 17:20:27.417557 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.417520    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="a8045934e1e6871bc6852c41e055ed6b990c7a3c6c4992f4726eecb157ae2942" exitCode=0
Jul 01 17:20:27.418010 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.417627    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"a8045934e1e6871bc6852c41e055ed6b990c7a3c6c4992f4726eecb157ae2942"}
Jul 01 17:20:27.419612 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.419565    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xfvp" event={"ID":"d86eb336-78a9-4647-9a6f-e426ca13af76","Type":"ContainerStarted","Data":"c7de115e9291fd81ba70bf50b914be56d1cb2225c7c31a50935c4de60041e078"}
Jul 01 17:20:27.419708 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.419640    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7xfvp" event={"ID":"d86eb336-78a9-4647-9a6f-e426ca13af76","Type":"ContainerStarted","Data":"4fb2a97fe819502f260596b8d2983c6464b8efe21008c5d9e5553b93d43cd4b1"}
Jul 01 17:20:27.419818 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.419799    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:27.464774 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:27.464694    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7xfvp" podStartSLOduration=252.63847431 podStartE2EDuration="4m14.464680642s" totalImagesPullingTime="1.826206332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:27.463957424 +0000 UTC m=+289.416383971" watchObservedRunningTime="2026-07-01 17:20:27.464680642 +0000 UTC m=+289.417107169"
Jul 01 17:20:29.321409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:29.321381    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86d6c8484-vxf2f"
Jul 01 17:20:29.437176 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:29.437153    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"35a9d8098b630a79c747e50056ac3214dfa2bd5489810f93aee7e20e22e0552a"}
Jul 01 17:20:30.444265 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:30.444231    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"e2cd237943578d17ca71943355487f685696c81fb0a3cf8ec36fe91e80f8e3b0"}
Jul 01 17:20:30.444671 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:30.444303    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"1530fb70c78ee6607cb9c57d66d43f0e618d53498cafd85a1595e6cce0b5c1e1"}
Jul 01 17:20:30.444671 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:30.444335    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"ff25e4f3411aa2c94fa92b39497c6f43a6b52386e1f8f71f2ae4c729faa891bb"}
Jul 01 17:20:30.444671 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:30.444362    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"e0eafa4d6e2f4f5706dd169958f5c5aa628b65f81546d3d36d0b916edad5f066"}
Jul 01 17:20:31.451683 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:31.451643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerStarted","Data":"ed8538b93c22d9489eab07f91fd7206bc026512990760ab4c4b8697285003a93"}
Jul 01 17:20:31.482115 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:31.482052    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.675034882 podStartE2EDuration="8.482031969s" totalImagesPullingTime="4.788246992s" totalInitContainerRuntime="18.750095ms" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:31.480853747 +0000 UTC m=+293.433280329" watchObservedRunningTime="2026-07-01 17:20:31.482031969 +0000 UTC m=+293.434458519"
Jul 01 17:20:34.542432 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:34.542400    2585 scope.go:117] "RemoveContainer" containerID="276cd0180d052a932434087157cef8e272adc222d868412ab0d8f86ce0f2de60"
Jul 01 17:20:35.227135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.227083    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-p8l82"]
Jul 01 17:20:35.229723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.229699    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:35.232620 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.232580    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 01 17:20:35.232914 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.232695    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-2smf4\""
Jul 01 17:20:35.232914 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.232696    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 01 17:20:35.240306 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.240282    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-p8l82"]
Jul 01 17:20:35.294940 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.294908    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjz88\" (UniqueName: \"kubernetes.io/projected/7babd4c8-1f91-454e-8ed7-0c6bf7938b40-kube-api-access-cjz88\") pod \"downloads-5499b6fb8f-p8l82\" (UID: \"7babd4c8-1f91-454e-8ed7-0c6bf7938b40\") " pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:35.395372 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.395333    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjz88\" (UniqueName: \"kubernetes.io/projected/7babd4c8-1f91-454e-8ed7-0c6bf7938b40-kube-api-access-cjz88\") pod \"downloads-5499b6fb8f-p8l82\" (UID: \"7babd4c8-1f91-454e-8ed7-0c6bf7938b40\") " pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:35.404431 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.404394    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjz88\" (UniqueName: \"kubernetes.io/projected/7babd4c8-1f91-454e-8ed7-0c6bf7938b40-kube-api-access-cjz88\") pod \"downloads-5499b6fb8f-p8l82\" (UID: \"7babd4c8-1f91-454e-8ed7-0c6bf7938b40\") " pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:35.467741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.467709    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:20:35.467887 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.467824    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" event={"ID":"1db61671-de63-445c-a5c0-ea12692510d0","Type":"ContainerStarted","Data":"7fdf2597574b9ccccd8f3568b0e6a5a761e7d2773a1349ea8085ba5eef42ebad"}
Jul 01 17:20:35.468109 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.468091    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:20:35.472686 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.472665    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn"
Jul 01 17:20:35.485528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.485459    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-q6xtn" podStartSLOduration=54.544174379 podStartE2EDuration="57.485445646s" totalImagesPullingTime="2.941271267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:19:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:35.485111205 +0000 UTC m=+297.437537752" watchObservedRunningTime="2026-07-01 17:20:35.485445646 +0000 UTC m=+297.437872191"
Jul 01 17:20:35.539498 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.539475    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:35.543675 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:20:35.543648    2585 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-2724j" podUID="ba2d162e-5b36-4378-a440-669ffe4fc9d4"
Jul 01 17:20:35.662152 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:35.662127    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-p8l82"]
Jul 01 17:20:35.663881 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:35.663857    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7babd4c8_1f91_454e_8ed7_0c6bf7938b40.slice/crio-02e6460da0a13c82761cf65a18d9dfb2763ab03ef7b85a425ea0e148a6c251f1 WatchSource:0}: Error finding container 02e6460da0a13c82761cf65a18d9dfb2763ab03ef7b85a425ea0e148a6c251f1: Status 404 returned error can't find the container with id 02e6460da0a13c82761cf65a18d9dfb2763ab03ef7b85a425ea0e148a6c251f1
Jul 01 17:20:36.474512 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:36.474475    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-p8l82" event={"ID":"7babd4c8-1f91-454e-8ed7-0c6bf7938b40","Type":"ContainerStarted","Data":"02e6460da0a13c82761cf65a18d9dfb2763ab03ef7b85a425ea0e148a6c251f1"}
Jul 01 17:20:37.431871 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:37.431841    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7xfvp"
Jul 01 17:20:38.472650 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:38.472585    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:20:38.474777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:38.474754    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:20:38.481510 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:38.481485    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:20:38.484758 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:38.484738    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:20:38.486025 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:38.486006    2585 kubelet.go:1648] "Image garbage collection succeeded"
Jul 01 17:20:46.248892 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.248856    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:20:46.253508 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.253483    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.256053 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.256033    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 01 17:20:46.256053 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.256047    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 01 17:20:46.257212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.257095    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 01 17:20:46.257212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.257117    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 01 17:20:46.257212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.257137    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dzjld\""
Jul 01 17:20:46.257212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.257105    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 01 17:20:46.263302 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.263281    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:20:46.403941 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.403900    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bgpb\" (UniqueName: \"kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.404116 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.403966    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.404116 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.404014    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.404116 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.404041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.404271 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.404117    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.404271 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.404159    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.504679 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504569    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.504859 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504770    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.504859 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504811    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.504972 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504877    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.504972 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.505064 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.504986    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bgpb\" (UniqueName: \"kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.505558 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.505528    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.505758 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.505736    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.505840 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.505736    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.507337 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.507313    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.507456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.507395    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.516674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.516649    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bgpb\" (UniqueName: \"kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb\") pod \"console-57948b8f7-p5zxs\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") " pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:46.565970 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:46.565941    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:20:49.542416 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:49.542385    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:20:53.000687 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.000665    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:20:53.003110 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:53.003078    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9704985_d1c8_43d2_acaa_ebf19d2a31a3.slice/crio-6ad4c47f80c70016aa1592c5f6922e46ceedbf051b5c08e8ee38451bb6811195 WatchSource:0}: Error finding container 6ad4c47f80c70016aa1592c5f6922e46ceedbf051b5c08e8ee38451bb6811195: Status 404 returned error can't find the container with id 6ad4c47f80c70016aa1592c5f6922e46ceedbf051b5c08e8ee38451bb6811195
Jul 01 17:20:53.005574 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.005557    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 17:20:53.548727 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.548643    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-p8l82" event={"ID":"7babd4c8-1f91-454e-8ed7-0c6bf7938b40","Type":"ContainerStarted","Data":"4eba9b64717e3f9cd397b8b85fda6267ea465dfe37d637459e5db58f5ea9db4b"}
Jul 01 17:20:53.548891 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.548791    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:53.550046 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.550023    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57948b8f7-p5zxs" event={"ID":"c9704985-d1c8-43d2-acaa-ebf19d2a31a3","Type":"ContainerStarted","Data":"6ad4c47f80c70016aa1592c5f6922e46ceedbf051b5c08e8ee38451bb6811195"}
Jul 01 17:20:53.550346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.550325    2585 patch_prober.go:28] interesting pod/downloads-5499b6fb8f-p8l82 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.19:8080/\": dial tcp 10.134.0.19:8080: connect: connection refused" start-of-body=
Jul 01 17:20:53.550414 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.550372    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-5499b6fb8f-p8l82" podUID="7babd4c8-1f91-454e-8ed7-0c6bf7938b40" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.19:8080/\": dial tcp 10.134.0.19:8080: connect: connection refused"
Jul 01 17:20:53.569487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:53.569445    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-p8l82" podStartSLOduration=0.984581532 podStartE2EDuration="18.56943219s" totalImagesPullingTime="17.584850658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:53.567240638 +0000 UTC m=+315.519667195" watchObservedRunningTime="2026-07-01 17:20:53.56943219 +0000 UTC m=+315.521858734"
Jul 01 17:20:54.567432 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:54.567399    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-p8l82"
Jul 01 17:20:55.529489 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.529439    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:20:55.539652 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.538968    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.551398 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.551370    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:20:55.554645 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.554490    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 01 17:20:55.589863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.589798    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.589863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.589850    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.590399 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.589878    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.590399 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.589948    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfqxx\" (UniqueName: \"kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.590776 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.589978    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.590776 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.590664    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.590776 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.590694    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692204 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfqxx\" (UniqueName: \"kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692214    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692241    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692269    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692369    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692398    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.692710 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.692645    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.694060 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.693345    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.694060 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.693747    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.694060 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.693834    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.694060 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.693959    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.695701 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.695680    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.695862 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.695840    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.701566 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.701541    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfqxx\" (UniqueName: \"kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx\") pod \"console-7d59f6cc8-xhjfw\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") " pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:55.869553 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:55.869468    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:20:56.645869 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:56.645824    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:20:56.648370 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:20:56.648334    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d009f3_1421_4e31_890b_85ee08c7d60e.slice/crio-1bbc7eb0ebe7baaa7491a874cc0cd2ed7e1113c7835d0370b7af9e2beee4c2f1 WatchSource:0}: Error finding container 1bbc7eb0ebe7baaa7491a874cc0cd2ed7e1113c7835d0370b7af9e2beee4c2f1: Status 404 returned error can't find the container with id 1bbc7eb0ebe7baaa7491a874cc0cd2ed7e1113c7835d0370b7af9e2beee4c2f1
Jul 01 17:20:57.574713 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:57.574628    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f6cc8-xhjfw" event={"ID":"08d009f3-1421-4e31-890b-85ee08c7d60e","Type":"ContainerStarted","Data":"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"}
Jul 01 17:20:57.574713 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:57.574690    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f6cc8-xhjfw" event={"ID":"08d009f3-1421-4e31-890b-85ee08c7d60e","Type":"ContainerStarted","Data":"1bbc7eb0ebe7baaa7491a874cc0cd2ed7e1113c7835d0370b7af9e2beee4c2f1"}
Jul 01 17:20:57.576472 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:57.576442    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57948b8f7-p5zxs" event={"ID":"c9704985-d1c8-43d2-acaa-ebf19d2a31a3","Type":"ContainerStarted","Data":"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"}
Jul 01 17:20:57.592389 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:57.592340    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d59f6cc8-xhjfw" podStartSLOduration=1.9317243309999999 podStartE2EDuration="2.592324504s" totalImagesPullingTime="660.600173ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:57.591902637 +0000 UTC m=+319.544329224" watchObservedRunningTime="2026-07-01 17:20:57.592324504 +0000 UTC m=+319.544751048"
Jul 01 17:20:57.608894 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:20:57.608844    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57948b8f7-p5zxs" podStartSLOduration=7.675304526 podStartE2EDuration="11.608827708s" totalImagesPullingTime="3.933523182s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:20:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:20:57.608087563 +0000 UTC m=+319.560514111" watchObservedRunningTime="2026-07-01 17:20:57.608827708 +0000 UTC m=+319.561254254"
Jul 01 17:21:05.869993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:05.869958    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:21:05.870451 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:05.870005    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:21:05.874565 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:05.874545    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:21:06.566908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.566880    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:06.567177 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.566934    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:06.571500 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.571478    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:06.613893 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.613868    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:06.614308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.614292    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:21:06.676062 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:06.676018    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:21:07.869390 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:07.869351    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/init-config-reloader/0.log"
Jul 01 17:21:07.876070 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:07.876050    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/alertmanager/0.log"
Jul 01 17:21:07.882867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:07.882842    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/config-reloader/0.log"
Jul 01 17:21:08.066138 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:08.066107    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/kube-rbac-proxy-web/0.log"
Jul 01 17:21:08.266148 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:08.266112    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/kube-rbac-proxy/0.log"
Jul 01 17:21:08.466405 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:08.466380    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/kube-rbac-proxy-metric/0.log"
Jul 01 17:21:08.667281 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:08.667200    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_82d40e72-803e-4d35-8bb8-a6b5e0985b46/prom-label-proxy/0.log"
Jul 01 17:21:09.066895 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:09.066865    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-state-metrics/0.log"
Jul 01 17:21:09.266440 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:09.266411    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-rbac-proxy-main/0.log"
Jul 01 17:21:09.466363 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:09.466292    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-rbac-proxy-self/0.log"
Jul 01 17:21:11.266325 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.266299    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/init-textfile/0.log"
Jul 01 17:21:11.467032 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.467007    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/node-exporter/0.log"
Jul 01 17:21:11.630320 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.630290    2585 generic.go:356] "Generic (PLEG): container finished" podID="8f167518-49f7-43b0-909b-c78a72de0220" containerID="72e9ea845848df9d703b08fe430fed10850387d971137f7ac4e37f1971fe1341" exitCode=0
Jul 01 17:21:11.630477 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.630366    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz" event={"ID":"8f167518-49f7-43b0-909b-c78a72de0220","Type":"ContainerDied","Data":"72e9ea845848df9d703b08fe430fed10850387d971137f7ac4e37f1971fe1341"}
Jul 01 17:21:11.630696 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.630682    2585 scope.go:117] "RemoveContainer" containerID="72e9ea845848df9d703b08fe430fed10850387d971137f7ac4e37f1971fe1341"
Jul 01 17:21:11.667880 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:11.667857    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/kube-rbac-proxy/0.log"
Jul 01 17:21:12.641021 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:12.640984    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-q6sgz" event={"ID":"8f167518-49f7-43b0-909b-c78a72de0220","Type":"ContainerStarted","Data":"1ddd2ff6ed6b6dd828aa5a788e3dcc7f6cda1b93c585bcfbeb4eec8239ea97e9"}
Jul 01 17:21:13.867738 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:13.867712    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-b8snh_11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d/prometheus-operator/0.log"
Jul 01 17:21:14.069824 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:14.069793    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-b8snh_11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d/kube-rbac-proxy/0.log"
Jul 01 17:21:16.466566 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:16.466537    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:21:16.669831 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:16.669798    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/3.log"
Jul 01 17:21:16.875818 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:16.875790    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-57948b8f7-p5zxs_c9704985-d1c8-43d2-acaa-ebf19d2a31a3/console/0.log"
Jul 01 17:21:17.073563 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:17.073537    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7d59f6cc8-xhjfw_08d009f3-1421-4e31-890b-85ee08c7d60e/console/0.log"
Jul 01 17:21:17.268332 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:17.268306    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-p8l82_7babd4c8-1f91-454e-8ed7-0c6bf7938b40/download-server/0.log"
Jul 01 17:21:33.638580 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:33.638487    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57948b8f7-p5zxs" podUID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" containerName="console" containerID="cri-o://e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc" gracePeriod=15
Jul 01 17:21:33.906097 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:33.906018    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:34.032614 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032554    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.032614 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032615    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.032861 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032663    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.032861 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032682    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bgpb\" (UniqueName: \"kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.032861 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032710    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.032861 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.032749    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca\") pod \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\" (UID: \"c9704985-d1c8-43d2-acaa-ebf19d2a31a3\") "
Jul 01 17:21:34.033115 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.033081    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:34.033235 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.033085    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config" (OuterVolumeSpecName: "console-config") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:34.033235 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.033165    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:34.034986 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.034959    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:34.035091 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.035063    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb" (OuterVolumeSpecName: "kube-api-access-2bgpb") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "kube-api-access-2bgpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:21:34.035091 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.035068    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c9704985-d1c8-43d2-acaa-ebf19d2a31a3" (UID: "c9704985-d1c8-43d2-acaa-ebf19d2a31a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:34.133731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133693    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-oauth-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.133731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133722    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bgpb\" (UniqueName: \"kubernetes.io/projected/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-kube-api-access-2bgpb\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.133731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133731    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.133731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133741    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-service-ca\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.134000 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133749    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-console-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.134000 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.133758    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9704985-d1c8-43d2-acaa-ebf19d2a31a3-oauth-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:34.729097 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.729064    2585 generic.go:356] "Generic (PLEG): container finished" podID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" containerID="e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc" exitCode=0
Jul 01 17:21:34.729546 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.729107    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57948b8f7-p5zxs"
Jul 01 17:21:34.729546 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.729155    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57948b8f7-p5zxs" event={"ID":"c9704985-d1c8-43d2-acaa-ebf19d2a31a3","Type":"ContainerDied","Data":"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"}
Jul 01 17:21:34.729546 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.729216    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57948b8f7-p5zxs" event={"ID":"c9704985-d1c8-43d2-acaa-ebf19d2a31a3","Type":"ContainerDied","Data":"6ad4c47f80c70016aa1592c5f6922e46ceedbf051b5c08e8ee38451bb6811195"}
Jul 01 17:21:34.729546 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.729247    2585 scope.go:117] "RemoveContainer" containerID="e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"
Jul 01 17:21:34.737737 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.737709    2585 scope.go:117] "RemoveContainer" containerID="e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"
Jul 01 17:21:34.738013 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:21:34.737994    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc\": container with ID starting with e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc not found: ID does not exist" containerID="e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"
Jul 01 17:21:34.738062 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.738022    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc"} err="failed to get container status \"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc\": rpc error: code = NotFound desc = could not find container \"e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc\": container with ID starting with e985d414c5dc91277bb1889e0fff944706a103889c01442006775a4d4025bacc not found: ID does not exist"
Jul 01 17:21:34.747222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.747199    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:21:34.751791 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:34.751772    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57948b8f7-p5zxs"]
Jul 01 17:21:36.546574 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:36.546538    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" path="/var/lib/kubelet/pods/c9704985-d1c8-43d2-acaa-ebf19d2a31a3/volumes"
Jul 01 17:21:42.594977 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.594942    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595241    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="alertmanager" containerID="cri-o://35a9d8098b630a79c747e50056ac3214dfa2bd5489810f93aee7e20e22e0552a" gracePeriod=120
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595313    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-web" containerID="cri-o://ff25e4f3411aa2c94fa92b39497c6f43a6b52386e1f8f71f2ae4c729faa891bb" gracePeriod=120
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595325    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy" containerID="cri-o://1530fb70c78ee6607cb9c57d66d43f0e618d53498cafd85a1595e6cce0b5c1e1" gracePeriod=120
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595384    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="config-reloader" containerID="cri-o://e0eafa4d6e2f4f5706dd169958f5c5aa628b65f81546d3d36d0b916edad5f066" gracePeriod=120
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595303    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-metric" containerID="cri-o://e2cd237943578d17ca71943355487f685696c81fb0a3cf8ec36fe91e80f8e3b0" gracePeriod=120
Jul 01 17:21:42.595493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.595404    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="prom-label-proxy" containerID="cri-o://ed8538b93c22d9489eab07f91fd7206bc026512990760ab4c4b8697285003a93" gracePeriod=120
Jul 01 17:21:42.762693 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762665    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="ed8538b93c22d9489eab07f91fd7206bc026512990760ab4c4b8697285003a93" exitCode=0
Jul 01 17:21:42.762693 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762689    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="1530fb70c78ee6607cb9c57d66d43f0e618d53498cafd85a1595e6cce0b5c1e1" exitCode=0
Jul 01 17:21:42.762693 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762697    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="e0eafa4d6e2f4f5706dd169958f5c5aa628b65f81546d3d36d0b916edad5f066" exitCode=0
Jul 01 17:21:42.762908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762705    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="35a9d8098b630a79c747e50056ac3214dfa2bd5489810f93aee7e20e22e0552a" exitCode=0
Jul 01 17:21:42.762908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762735    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"ed8538b93c22d9489eab07f91fd7206bc026512990760ab4c4b8697285003a93"}
Jul 01 17:21:42.762908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762796    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"1530fb70c78ee6607cb9c57d66d43f0e618d53498cafd85a1595e6cce0b5c1e1"}
Jul 01 17:21:42.762908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762823    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"e0eafa4d6e2f4f5706dd169958f5c5aa628b65f81546d3d36d0b916edad5f066"}
Jul 01 17:21:42.762908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:42.762847    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"35a9d8098b630a79c747e50056ac3214dfa2bd5489810f93aee7e20e22e0552a"}
Jul 01 17:21:43.771394 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.771363    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="e2cd237943578d17ca71943355487f685696c81fb0a3cf8ec36fe91e80f8e3b0" exitCode=0
Jul 01 17:21:43.771394 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.771391    2585 generic.go:356] "Generic (PLEG): container finished" podID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerID="ff25e4f3411aa2c94fa92b39497c6f43a6b52386e1f8f71f2ae4c729faa891bb" exitCode=0
Jul 01 17:21:43.771752 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.771447    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"e2cd237943578d17ca71943355487f685696c81fb0a3cf8ec36fe91e80f8e3b0"}
Jul 01 17:21:43.771752 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.771514    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"ff25e4f3411aa2c94fa92b39497c6f43a6b52386e1f8f71f2ae4c729faa891bb"}
Jul 01 17:21:43.837371 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.837350    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:43.908851 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908774    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.908851 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908817    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.908851 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908844    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908875    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908902    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjv4k\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908941    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908962    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.908990    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909017    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909051    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909108 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909078    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909490 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909115    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909490 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909155    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca\") pod \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\" (UID: \"82d40e72-803e-4d35-8bb8-a6b5e0985b46\") "
Jul 01 17:21:43.909490 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909204    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:43.909682 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909570    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:21:43.909786 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909762    2585 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-metrics-client-ca\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:43.909847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909796    2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:43.909982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.909960    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:21:43.912691 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.912657    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out" (OuterVolumeSpecName: "config-out") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:21:43.913399 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.913366    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.913543 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.913495    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k" (OuterVolumeSpecName: "kube-api-access-hjv4k") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "kube-api-access-hjv4k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:21:43.913639 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.913571    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.913823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.913786    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.914228 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.914073    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:21:43.914228 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.914153    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume" (OuterVolumeSpecName: "config-volume") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.914445 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.914424    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.917078 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.917018    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:43.923028 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:43.923005    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config" (OuterVolumeSpecName: "web-config") pod "82d40e72-803e-4d35-8bb8-a6b5e0985b46" (UID: "82d40e72-803e-4d35-8bb8-a6b5e0985b46"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:21:44.010184 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010135    2585 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-tls-assets\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010184 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010176    2585 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-cluster-tls-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010184 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010190    2585 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-alertmanager-main-db\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010204    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010219    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010232    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-main-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010243    2585 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-volume\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010253    2585 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-web-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010265    2585 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/82d40e72-803e-4d35-8bb8-a6b5e0985b46-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010276    2585 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/82d40e72-803e-4d35-8bb8-a6b5e0985b46-config-out\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.010429 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.010287    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjv4k\" (UniqueName: \"kubernetes.io/projected/82d40e72-803e-4d35-8bb8-a6b5e0985b46-kube-api-access-hjv4k\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:21:44.778703 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.778671    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"82d40e72-803e-4d35-8bb8-a6b5e0985b46","Type":"ContainerDied","Data":"65149ad03949dae32e45461e4ede56cca57fb44b79a7af20ed54269068a2f35f"}
Jul 01 17:21:44.779119 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.778715    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.779119 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.778747    2585 scope.go:117] "RemoveContainer" containerID="ed8538b93c22d9489eab07f91fd7206bc026512990760ab4c4b8697285003a93"
Jul 01 17:21:44.786525 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.786508    2585 scope.go:117] "RemoveContainer" containerID="e2cd237943578d17ca71943355487f685696c81fb0a3cf8ec36fe91e80f8e3b0"
Jul 01 17:21:44.797810 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.797786    2585 scope.go:117] "RemoveContainer" containerID="1530fb70c78ee6607cb9c57d66d43f0e618d53498cafd85a1595e6cce0b5c1e1"
Jul 01 17:21:44.804226 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.804165    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:44.805032 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.805000    2585 scope.go:117] "RemoveContainer" containerID="ff25e4f3411aa2c94fa92b39497c6f43a6b52386e1f8f71f2ae4c729faa891bb"
Jul 01 17:21:44.810338 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.810320    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:44.811708 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.811689    2585 scope.go:117] "RemoveContainer" containerID="e0eafa4d6e2f4f5706dd169958f5c5aa628b65f81546d3d36d0b916edad5f066"
Jul 01 17:21:44.818260 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.818243    2585 scope.go:117] "RemoveContainer" containerID="35a9d8098b630a79c747e50056ac3214dfa2bd5489810f93aee7e20e22e0552a"
Jul 01 17:21:44.824862 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.824847    2585 scope.go:117] "RemoveContainer" containerID="a8045934e1e6871bc6852c41e055ed6b990c7a3c6c4992f4726eecb157ae2942"
Jul 01 17:21:44.847744 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.847720    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:44.848000 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.847988    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="alertmanager"
Jul 01 17:21:44.848040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848001    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="alertmanager"
Jul 01 17:21:44.848040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848013    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="init-config-reloader"
Jul 01 17:21:44.848040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848019    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="init-config-reloader"
Jul 01 17:21:44.848040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848028    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-metric"
Jul 01 17:21:44.848040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848033    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-metric"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848044    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="prom-label-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848049    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="prom-label-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848057    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-web"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848064    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-web"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848070    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" containerName="console"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848075    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" containerName="console"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848080    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="config-reloader"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848085    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="config-reloader"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848092    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848096    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848138    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848145    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9704985-d1c8-43d2-acaa-ebf19d2a31a3" containerName="console"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848152    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="config-reloader"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848158    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-metric"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848163    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="prom-label-proxy"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848168    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="alertmanager"
Jul 01 17:21:44.848222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.848173    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" containerName="kube-rbac-proxy-web"
Jul 01 17:21:44.851873 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.851859    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.856529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.856509    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 01 17:21:44.856672 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.856503    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 01 17:21:44.856672 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.856645    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 01 17:21:44.856804 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.856754    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 01 17:21:44.857151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.857012    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-l7bm9\""
Jul 01 17:21:44.857151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.857025    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 01 17:21:44.857151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.857102    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 01 17:21:44.857151 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.857131    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 01 17:21:44.857656 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.857573    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 01 17:21:44.867034 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.867018    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 01 17:21:44.869667 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.869646    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:44.918014 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.917983    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918014 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918017    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918041    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918096    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-volume\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918128    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918172    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918192    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918232    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-web-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918256    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918275    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkb8b\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-kube-api-access-zkb8b\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918301    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-out\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918327    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:44.918355 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:44.918343    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018651 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018618    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018651 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018651    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018677    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018791    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018842    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.018899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.018875    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-volume\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019105 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019049    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019159 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019139    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019167    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019212 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019205    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-web-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019243    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019267    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkb8b\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-kube-api-access-zkb8b\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019295    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-out\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.019655 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019626    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.021772 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.021745    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.021772 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.021760    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-out\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.021956 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.021810    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.021956 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.019770    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.021956 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.021810    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-config-volume\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.022118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.022036    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.022361 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.022317    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.022958 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.022929    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.023184 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.023161    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.023862 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.023845    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.023954 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.023937    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-web-config\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.032567 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.032535    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkb8b\" (UniqueName: \"kubernetes.io/projected/c73888bd-4b3a-4b08-9c4e-77c452e1cff2-kube-api-access-zkb8b\") pod \"alertmanager-main-0\" (UID: \"c73888bd-4b3a-4b08-9c4e-77c452e1cff2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.161735 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.161661    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 01 17:21:45.306411 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.306390    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 01 17:21:45.308607 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:21:45.308565    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc73888bd_4b3a_4b08_9c4e_77c452e1cff2.slice/crio-de4f84d65df8c93443c956f4d2f8512c53ef204776bc6044ec1b66f6dda5195d WatchSource:0}: Error finding container de4f84d65df8c93443c956f4d2f8512c53ef204776bc6044ec1b66f6dda5195d: Status 404 returned error can't find the container with id de4f84d65df8c93443c956f4d2f8512c53ef204776bc6044ec1b66f6dda5195d
Jul 01 17:21:45.784036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.784007    2585 generic.go:356] "Generic (PLEG): container finished" podID="c73888bd-4b3a-4b08-9c4e-77c452e1cff2" containerID="538de0828773b5f61e1da5c02cb86ba0ddaf87a542ec8419bbbbc1dc34e33435" exitCode=0
Jul 01 17:21:45.784379 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.784043    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerDied","Data":"538de0828773b5f61e1da5c02cb86ba0ddaf87a542ec8419bbbbc1dc34e33435"}
Jul 01 17:21:45.784379 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:45.784094    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"de4f84d65df8c93443c956f4d2f8512c53ef204776bc6044ec1b66f6dda5195d"}
Jul 01 17:21:46.546504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.546471    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82d40e72-803e-4d35-8bb8-a6b5e0985b46" path="/var/lib/kubelet/pods/82d40e72-803e-4d35-8bb8-a6b5e0985b46/volumes"
Jul 01 17:21:46.793568 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793534    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"689db7160f0679d0fbeab7cf867188c6ef47d7eeb27275a2017b7a04ca923cfb"}
Jul 01 17:21:46.794013 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793669    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"c42a47dbabc063342f52201bf2903eef928bd284a7d8d759237a275c8a1fc0ec"}
Jul 01 17:21:46.794013 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793722    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"6204ef7a84aab7f5dfc64f51bcf2e88b84d16d4fc4967ff4beefb8e001dd6cca"}
Jul 01 17:21:46.794013 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793764    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"99c0d5b9c8b99568e7013c17ec7d8e23645d61fd0b1a28e5e375223ce048fc48"}
Jul 01 17:21:46.794013 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"0765f30ffc2506abf8fbfc9f230e31a9901d10922001ce59ed2b4902450aed1a"}
Jul 01 17:21:46.794013 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.793845    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c73888bd-4b3a-4b08-9c4e-77c452e1cff2","Type":"ContainerStarted","Data":"427683716bd2a01d9d1af43ae9fbb3a8d0375e660ee0dd773fa4a9aab9707dec"}
Jul 01 17:21:46.842219 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:46.842112    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.827106773 podStartE2EDuration="2.842096863s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.99009ms" isStatefulPod=false podCreationTimestamp="2026-07-01 17:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:21:46.840645538 +0000 UTC m=+368.793072085" watchObservedRunningTime="2026-07-01 17:21:46.842096863 +0000 UTC m=+368.794523413"
Jul 01 17:21:50.007553 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.007518    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:21:50.010908 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.010883    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.024138 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.024112    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:21:50.063033 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063004    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063053    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063131    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063167    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063287 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063185    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063287 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063214    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.063287 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.063273    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99vzk\" (UniqueName: \"kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164620 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164563    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164718    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164740    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164756    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164938 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164779    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164938 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164818    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99vzk\" (UniqueName: \"kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.164938 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.164860    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.165453 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.165427    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.165579 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.165542    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.165579 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.165556    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.165579 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.165542    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.167252 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.167226    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.167351 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.167273    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.172914 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.172893    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99vzk\" (UniqueName: \"kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk\") pod \"console-7fc6465465-2qv5q\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") " pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.320658 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.320569    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:21:50.453061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.453029    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:21:50.455267 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:21:50.455238    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc641ef60_3165_4542_bd02_149635f93624.slice/crio-b5089e786dfbb322d1d07bd5a3a82814173f84e2ad1cc469951f07f4ffbe0976 WatchSource:0}: Error finding container b5089e786dfbb322d1d07bd5a3a82814173f84e2ad1cc469951f07f4ffbe0976: Status 404 returned error can't find the container with id b5089e786dfbb322d1d07bd5a3a82814173f84e2ad1cc469951f07f4ffbe0976
Jul 01 17:21:50.812846 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.812807    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6465465-2qv5q" event={"ID":"c641ef60-3165-4542-bd02-149635f93624","Type":"ContainerStarted","Data":"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"}
Jul 01 17:21:50.813012 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.812859    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6465465-2qv5q" event={"ID":"c641ef60-3165-4542-bd02-149635f93624","Type":"ContainerStarted","Data":"b5089e786dfbb322d1d07bd5a3a82814173f84e2ad1cc469951f07f4ffbe0976"}
Jul 01 17:21:50.831036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:50.830986    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7fc6465465-2qv5q" podStartSLOduration=1.830972562 podStartE2EDuration="1.830972562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:21:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:21:50.830294558 +0000 UTC m=+372.782721103" watchObservedRunningTime="2026-07-01 17:21:50.830972562 +0000 UTC m=+372.783399107"
Jul 01 17:21:52.381668 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.381632    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:21:52.383825 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.383808    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ba2d162e-5b36-4378-a440-669ffe4fc9d4-metrics-certs\") pod \"network-metrics-daemon-2724j\" (UID: \"ba2d162e-5b36-4378-a440-669ffe4fc9d4\") " pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:21:52.546354 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.546329    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnmdh\""
Jul 01 17:21:52.554641 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.554617    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2724j"
Jul 01 17:21:52.680930 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.680879    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2724j"]
Jul 01 17:21:52.683017 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:21:52.682989    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba2d162e_5b36_4378_a440_669ffe4fc9d4.slice/crio-2d69c5a27c1a56a67cbcfaad36d1ed4d59d4729bb6f030867e5108d953139525 WatchSource:0}: Error finding container 2d69c5a27c1a56a67cbcfaad36d1ed4d59d4729bb6f030867e5108d953139525: Status 404 returned error can't find the container with id 2d69c5a27c1a56a67cbcfaad36d1ed4d59d4729bb6f030867e5108d953139525
Jul 01 17:21:52.822114 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:52.822080    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2724j" event={"ID":"ba2d162e-5b36-4378-a440-669ffe4fc9d4","Type":"ContainerStarted","Data":"2d69c5a27c1a56a67cbcfaad36d1ed4d59d4729bb6f030867e5108d953139525"}
Jul 01 17:21:53.828478 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:53.828427    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2724j" event={"ID":"ba2d162e-5b36-4378-a440-669ffe4fc9d4","Type":"ContainerStarted","Data":"cd6d78de8f79cf6ebdb24514bcbe00fb81e1a95aebc85c00c734cc28ba1bb7f6"}
Jul 01 17:21:53.828872 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:53.828483    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2724j" event={"ID":"ba2d162e-5b36-4378-a440-669ffe4fc9d4","Type":"ContainerStarted","Data":"fc7cc8c11d848f59d081c347ebfc1c7dd9f7ffcea424072bef5b2732fcf9fb7d"}
Jul 01 17:21:53.852040 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:21:53.851990    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2724j" podStartSLOduration=374.891226233 podStartE2EDuration="6m15.851972407s" totalImagesPullingTime="960.746174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:15:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:21:53.847928831 +0000 UTC m=+375.800355377" watchObservedRunningTime="2026-07-01 17:21:53.851972407 +0000 UTC m=+375.804398951"
Jul 01 17:22:00.320965 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:00.320916    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:22:00.321356 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:00.320993    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:22:00.325722 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:00.325697    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:22:00.863540 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:00.863511    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:22:00.912310 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:00.912275    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:22:25.936538 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:25.936472    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7d59f6cc8-xhjfw" podUID="08d009f3-1421-4e31-890b-85ee08c7d60e" containerName="console" containerID="cri-o://acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e" gracePeriod=15
Jul 01 17:22:26.175442 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.175419    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:22:26.273427 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273400    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273609 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273449    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273609 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273482    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273697 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273640    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273697 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273678    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfqxx\" (UniqueName: \"kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273782 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273711    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273822 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273797    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca\") pod \"08d009f3-1421-4e31-890b-85ee08c7d60e\" (UID: \"08d009f3-1421-4e31-890b-85ee08c7d60e\") "
Jul 01 17:22:26.273858 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273832    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config" (OuterVolumeSpecName: "console-config") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:22:26.273858 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.273848    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:22:26.274090 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.274069    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-console-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.274188 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.274093    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-trusted-ca-bundle\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.274234 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.274190    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:22:26.274234 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.274196    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca" (OuterVolumeSpecName: "service-ca") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:22:26.275955 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.275918    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx" (OuterVolumeSpecName: "kube-api-access-mfqxx") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "kube-api-access-mfqxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:22:26.276058 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.275957    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:22:26.276058 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.275973    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "08d009f3-1421-4e31-890b-85ee08c7d60e" (UID: "08d009f3-1421-4e31-890b-85ee08c7d60e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:22:26.375118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.375085    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-oauth-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.375118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.375108    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfqxx\" (UniqueName: \"kubernetes.io/projected/08d009f3-1421-4e31-890b-85ee08c7d60e-kube-api-access-mfqxx\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.375118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.375118    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-oauth-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.375118 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.375126    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08d009f3-1421-4e31-890b-85ee08c7d60e-service-ca\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.375358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.375134    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08d009f3-1421-4e31-890b-85ee08c7d60e-console-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:22:26.965294 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.965207    2585 generic.go:356] "Generic (PLEG): container finished" podID="08d009f3-1421-4e31-890b-85ee08c7d60e" containerID="acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e" exitCode=0
Jul 01 17:22:26.965294 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.965255    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d59f6cc8-xhjfw"
Jul 01 17:22:26.965717 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.965294    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f6cc8-xhjfw" event={"ID":"08d009f3-1421-4e31-890b-85ee08c7d60e","Type":"ContainerDied","Data":"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"}
Jul 01 17:22:26.965717 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.965348    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d59f6cc8-xhjfw" event={"ID":"08d009f3-1421-4e31-890b-85ee08c7d60e","Type":"ContainerDied","Data":"1bbc7eb0ebe7baaa7491a874cc0cd2ed7e1113c7835d0370b7af9e2beee4c2f1"}
Jul 01 17:22:26.965717 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.965370    2585 scope.go:117] "RemoveContainer" containerID="acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"
Jul 01 17:22:26.973487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.973469    2585 scope.go:117] "RemoveContainer" containerID="acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"
Jul 01 17:22:26.973770 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:22:26.973747    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e\": container with ID starting with acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e not found: ID does not exist" containerID="acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"
Jul 01 17:22:26.973869 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.973780    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e"} err="failed to get container status \"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e\": rpc error: code = NotFound desc = could not find container \"acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e\": container with ID starting with acca7042051a37227b7be9081cc059c76aad5eb49cd217bf4ac3b405d68a899e not found: ID does not exist"
Jul 01 17:22:26.982939 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.982914    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:22:26.987284 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:26.987265    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d59f6cc8-xhjfw"]
Jul 01 17:22:28.547449 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:28.547417    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08d009f3-1421-4e31-890b-85ee08c7d60e" path="/var/lib/kubelet/pods/08d009f3-1421-4e31-890b-85ee08c7d60e/volumes"
Jul 01 17:22:50.771784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.771748    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"]
Jul 01 17:22:50.772147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.772033    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08d009f3-1421-4e31-890b-85ee08c7d60e" containerName="console"
Jul 01 17:22:50.772147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.772044    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="08d009f3-1421-4e31-890b-85ee08c7d60e" containerName="console"
Jul 01 17:22:50.772147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.772123    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="08d009f3-1421-4e31-890b-85ee08c7d60e" containerName="console"
Jul 01 17:22:50.775169 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.775151    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.778167 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.778146    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 01 17:22:50.778295 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.778150    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 01 17:22:50.779213 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.779193    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-cb9g2\""
Jul 01 17:22:50.786496 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.786478    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"]
Jul 01 17:22:50.864149 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.864117    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.864330 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.864160    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd7cx\" (UniqueName: \"kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.864330 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.864242    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.964982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.964941    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.965134 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.964997    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd7cx\" (UniqueName: \"kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.965134 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.965029    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.965315 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.965294    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.965381 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.965354    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:50.984196 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:50.984170    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd7cx\" (UniqueName: \"kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:51.084097 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:51.084012    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:22:51.213232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:51.213207    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"]
Jul 01 17:22:51.214770 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:22:51.214745    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ea6f052_dac1_480c_8bb0_66ed28c40c36.slice/crio-6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768 WatchSource:0}: Error finding container 6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768: Status 404 returned error can't find the container with id 6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768
Jul 01 17:22:52.073420 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:52.073372    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerStarted","Data":"6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768"}
Jul 01 17:22:59.106032 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:59.105999    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerID="15e3c09b2240d03972d572759fc5a7ac3dd4416ed278b4447ea4bd0492d25e02" exitCode=0
Jul 01 17:22:59.106439 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:22:59.106093    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerDied","Data":"15e3c09b2240d03972d572759fc5a7ac3dd4416ed278b4447ea4bd0492d25e02"}
Jul 01 17:23:02.121079 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:02.121040    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerID="f8c693fde986a7dd8311570799e49c9220d1c05781fda6434b5bf2f0ac552755" exitCode=0
Jul 01 17:23:02.121444 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:02.121094    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerDied","Data":"f8c693fde986a7dd8311570799e49c9220d1c05781fda6434b5bf2f0ac552755"}
Jul 01 17:23:10.157007 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:10.156970    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerStarted","Data":"c3888fa6bba128233b5a09c5242703f95e7b0beef505838feb5f7d05783e42b4"}
Jul 01 17:23:10.183325 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:10.183284    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" podStartSLOduration=2.485079834 podStartE2EDuration="20.183267561s" totalImagesPullingTime="17.698187727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:22:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:23:10.181975927 +0000 UTC m=+452.134402474" watchObservedRunningTime="2026-07-01 17:23:10.183267561 +0000 UTC m=+452.135694106"
Jul 01 17:23:11.162877 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:11.162840    2585 generic.go:356] "Generic (PLEG): container finished" podID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerID="c3888fa6bba128233b5a09c5242703f95e7b0beef505838feb5f7d05783e42b4" exitCode=0
Jul 01 17:23:11.163255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:11.162909    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerDied","Data":"c3888fa6bba128233b5a09c5242703f95e7b0beef505838feb5f7d05783e42b4"}
Jul 01 17:23:12.292892 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.292867    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:23:12.346816 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.346792    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle\") pod \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") "
Jul 01 17:23:12.346971 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.346825    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd7cx\" (UniqueName: \"kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx\") pod \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") "
Jul 01 17:23:12.346971 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.346861    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util\") pod \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\" (UID: \"7ea6f052-dac1-480c-8bb0-66ed28c40c36\") "
Jul 01 17:23:12.347438 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.347413    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle" (OuterVolumeSpecName: "bundle") pod "7ea6f052-dac1-480c-8bb0-66ed28c40c36" (UID: "7ea6f052-dac1-480c-8bb0-66ed28c40c36"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:23:12.349017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.348990    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx" (OuterVolumeSpecName: "kube-api-access-dd7cx") pod "7ea6f052-dac1-480c-8bb0-66ed28c40c36" (UID: "7ea6f052-dac1-480c-8bb0-66ed28c40c36"). InnerVolumeSpecName "kube-api-access-dd7cx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:23:12.351140 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.351117    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util" (OuterVolumeSpecName: "util") pod "7ea6f052-dac1-480c-8bb0-66ed28c40c36" (UID: "7ea6f052-dac1-480c-8bb0-66ed28c40c36"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:23:12.447864 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.447781    2585 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-util\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:23:12.447864 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.447843    2585 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7ea6f052-dac1-480c-8bb0-66ed28c40c36-bundle\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:23:12.447864 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:12.447856    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd7cx\" (UniqueName: \"kubernetes.io/projected/7ea6f052-dac1-480c-8bb0-66ed28c40c36-kube-api-access-dd7cx\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:23:13.173269 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:13.173226    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4" event={"ID":"7ea6f052-dac1-480c-8bb0-66ed28c40c36","Type":"ContainerDied","Data":"6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768"}
Jul 01 17:23:13.173269 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:13.173275    2585 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6acc49fa1c2a2c5a7c5b40222ec5b687374186f51d05d78bc89fc8bd09cb7768"
Jul 01 17:23:13.173482 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:13.173239    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589gbcd4"
Jul 01 17:23:23.299360 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299322    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"]
Jul 01 17:23:23.299864 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299848    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="extract"
Jul 01 17:23:23.299909 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299868    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="extract"
Jul 01 17:23:23.299909 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299883    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="pull"
Jul 01 17:23:23.299909 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299891    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="pull"
Jul 01 17:23:23.299909 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299906    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="util"
Jul 01 17:23:23.300027 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.299913    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="util"
Jul 01 17:23:23.300027 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.300009    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ea6f052-dac1-480c-8bb0-66ed28c40c36" containerName="extract"
Jul 01 17:23:23.304968 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.304946    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.307692 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.307666    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 01 17:23:23.307692 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.307683    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 01 17:23:23.308469 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.308450    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\""
Jul 01 17:23:23.308469 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.308465    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-hf5nl\""
Jul 01 17:23:23.308629 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.308497    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 01 17:23:23.308629 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.308530    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\""
Jul 01 17:23:23.314538 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.314510    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"]
Jul 01 17:23:23.438188 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.438149    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1074ed0e-817e-4aef-81a7-d927e03f8139-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.438409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.438208    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd5hz\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-kube-api-access-zd5hz\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.438409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.438381    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-certificates\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.488442 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.488412    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-9qtjx"]
Jul 01 17:23:23.490665 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.490650    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.493418 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.493398    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 01 17:23:23.502750 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.502729    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-9qtjx"]
Jul 01 17:23:23.539787 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.539756    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-certificates\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.539922 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.539836    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1074ed0e-817e-4aef-81a7-d927e03f8139-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.539922 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.539875    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd5hz\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-kube-api-access-zd5hz\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.540268 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.540243    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1074ed0e-817e-4aef-81a7-d927e03f8139-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.543091 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.543066    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-certificates\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.549192 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.549171    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd5hz\" (UniqueName: \"kubernetes.io/projected/1074ed0e-817e-4aef-81a7-d927e03f8139-kube-api-access-zd5hz\") pod \"keda-metrics-apiserver-559f75f947-qx5v6\" (UID: \"1074ed0e-817e-4aef-81a7-d927e03f8139\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.617453 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.617349    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:23.640418 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.640381    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vrsr\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-kube-api-access-8vrsr\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.640616 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.640575    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-certificates\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.741507 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.741476    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vrsr\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-kube-api-access-8vrsr\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.741707 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.741579    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-certificates\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.744275 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.744236    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-certificates\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.752154 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.752130    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vrsr\" (UniqueName: \"kubernetes.io/projected/79a4f6c9-2f7d-492d-a20b-2052b24f5f44-kube-api-access-8vrsr\") pod \"keda-admission-5fbd5c4644-9qtjx\" (UID: \"79a4f6c9-2f7d-492d-a20b-2052b24f5f44\") " pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.765963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.765884    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"]
Jul 01 17:23:23.768679 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:23:23.768649    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1074ed0e_817e_4aef_81a7_d927e03f8139.slice/crio-13891b7fb0663f51c11aacc60fb5644819d223a22f98482bc6d6fdeabd40d0c6 WatchSource:0}: Error finding container 13891b7fb0663f51c11aacc60fb5644819d223a22f98482bc6d6fdeabd40d0c6: Status 404 returned error can't find the container with id 13891b7fb0663f51c11aacc60fb5644819d223a22f98482bc6d6fdeabd40d0c6
Jul 01 17:23:23.801366 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.801336    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:23.947208 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:23.947176    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-9qtjx"]
Jul 01 17:23:23.950110 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:23:23.950086    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79a4f6c9_2f7d_492d_a20b_2052b24f5f44.slice/crio-fa82c3e23114c8b669e8ae15d176934ce42de1caa9e5d89c384e57c11650caba WatchSource:0}: Error finding container fa82c3e23114c8b669e8ae15d176934ce42de1caa9e5d89c384e57c11650caba: Status 404 returned error can't find the container with id fa82c3e23114c8b669e8ae15d176934ce42de1caa9e5d89c384e57c11650caba
Jul 01 17:23:24.224007 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:24.223906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx" event={"ID":"79a4f6c9-2f7d-492d-a20b-2052b24f5f44","Type":"ContainerStarted","Data":"fa82c3e23114c8b669e8ae15d176934ce42de1caa9e5d89c384e57c11650caba"}
Jul 01 17:23:24.225000 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:24.224976    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6" event={"ID":"1074ed0e-817e-4aef-81a7-d927e03f8139","Type":"ContainerStarted","Data":"13891b7fb0663f51c11aacc60fb5644819d223a22f98482bc6d6fdeabd40d0c6"}
Jul 01 17:23:27.239634 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:27.239577    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx" event={"ID":"79a4f6c9-2f7d-492d-a20b-2052b24f5f44","Type":"ContainerStarted","Data":"020cfeb72878c060ff5d2ef26d0efd44f0a416368adb6d6b70450cc2c6c5f15e"}
Jul 01 17:23:27.240104 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:27.239712    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:23:27.260039 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:27.259978    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx" podStartSLOduration=2.001232275 podStartE2EDuration="4.259955866s" totalImagesPullingTime="2.258723591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:23:27.258015737 +0000 UTC m=+469.210442283" watchObservedRunningTime="2026-07-01 17:23:27.259955866 +0000 UTC m=+469.212382413"
Jul 01 17:23:28.244609 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:28.244553    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6" event={"ID":"1074ed0e-817e-4aef-81a7-d927e03f8139","Type":"ContainerStarted","Data":"e3de202c5038b7a267ead5b6811261c9ddb67d76e8736ba43582ae6da565f138"}
Jul 01 17:23:28.244981 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:28.244801    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:28.267731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:28.267674    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6" podStartSLOduration=1.372089632 podStartE2EDuration="5.267655611s" totalImagesPullingTime="3.895565979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:23:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:23:28.26674061 +0000 UTC m=+470.219167157" watchObservedRunningTime="2026-07-01 17:23:28.267655611 +0000 UTC m=+470.220082158"
Jul 01 17:23:39.254087 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:39.254054    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-qx5v6"
Jul 01 17:23:48.246517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:23:48.246481    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-9qtjx"
Jul 01 17:24:29.778866 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.778781    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:24:29.785233 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.785213    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:29.787991 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.787972    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 01 17:24:29.788434 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.788412    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 01 17:24:29.788571 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.788510    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\""
Jul 01 17:24:29.789184 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.789164    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-fbjpk\""
Jul 01 17:24:29.796084 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.796062    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:24:29.861899 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.861869    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:29.862081 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.861927    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74stg\" (UniqueName: \"kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:29.962363 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.962330    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74stg\" (UniqueName: \"kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:29.962535 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.962493    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:29.962641 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:24:29.962626    2585 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 01 17:24:29.962699 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:24:29.962688    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert podName:7c5f9e33-0ade-4708-b240-676540d5322f nodeName:}" failed. No retries permitted until 2026-07-01 17:24:30.462672167 +0000 UTC m=+532.415098691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert") pod "kserve-controller-manager-848c499849-fxn48" (UID: "7c5f9e33-0ade-4708-b240-676540d5322f") : secret "kserve-webhook-server-cert" not found
Jul 01 17:24:29.974061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:29.974037    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74stg\" (UniqueName: \"kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:30.471488 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:30.467344    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:30.473837 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:30.473805    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") pod \"kserve-controller-manager-848c499849-fxn48\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") " pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:30.697209 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:30.697168    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:30.826392 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:30.826362    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:24:30.829458 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:24:30.829430    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c5f9e33_0ade_4708_b240_676540d5322f.slice/crio-5ecfecf64e4ad6efb17a95e396cb60834ecd62d919dfff4dd52950c3fe238897 WatchSource:0}: Error finding container 5ecfecf64e4ad6efb17a95e396cb60834ecd62d919dfff4dd52950c3fe238897: Status 404 returned error can't find the container with id 5ecfecf64e4ad6efb17a95e396cb60834ecd62d919dfff4dd52950c3fe238897
Jul 01 17:24:31.504046 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:31.504005    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-fxn48" event={"ID":"7c5f9e33-0ade-4708-b240-676540d5322f","Type":"ContainerStarted","Data":"5ecfecf64e4ad6efb17a95e396cb60834ecd62d919dfff4dd52950c3fe238897"}
Jul 01 17:24:33.514214 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:33.514182    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-fxn48" event={"ID":"7c5f9e33-0ade-4708-b240-676540d5322f","Type":"ContainerStarted","Data":"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"}
Jul 01 17:24:33.514588 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:33.514321    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:24:33.532149 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:24:33.532107    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-848c499849-fxn48" podStartSLOduration=2.066187709 podStartE2EDuration="4.532090893s" totalImagesPullingTime="2.465903184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:24:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:24:33.530392768 +0000 UTC m=+535.482819315" watchObservedRunningTime="2026-07-01 17:24:33.532090893 +0000 UTC m=+535.484517438"
Jul 01 17:25:04.524291 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:04.524259    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:25:05.315478 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.315442    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:25:05.315680 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.315638    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-848c499849-fxn48" podUID="7c5f9e33-0ade-4708-b240-676540d5322f" containerName="manager" containerID="cri-o://3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1" gracePeriod=10
Jul 01 17:25:05.339795 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.339773    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-848c499849-bcgpt"]
Jul 01 17:25:05.342817 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.342803    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.352223 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.352200    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-848c499849-bcgpt"]
Jul 01 17:25:05.435176 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.435153    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcbnj\" (UniqueName: \"kubernetes.io/projected/0c645c55-8652-4a51-9bc1-dc2aeeb27549-kube-api-access-lcbnj\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.435267 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.435203    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c645c55-8652-4a51-9bc1-dc2aeeb27549-cert\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.535940 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.535911    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcbnj\" (UniqueName: \"kubernetes.io/projected/0c645c55-8652-4a51-9bc1-dc2aeeb27549-kube-api-access-lcbnj\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.536272 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.535968    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c645c55-8652-4a51-9bc1-dc2aeeb27549-cert\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.538225 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.538193    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0c645c55-8652-4a51-9bc1-dc2aeeb27549-cert\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.544297 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.544271    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcbnj\" (UniqueName: \"kubernetes.io/projected/0c645c55-8652-4a51-9bc1-dc2aeeb27549-kube-api-access-lcbnj\") pod \"kserve-controller-manager-848c499849-bcgpt\" (UID: \"0c645c55-8652-4a51-9bc1-dc2aeeb27549\") " pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.564420 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.564397    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:25:05.636368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636288    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74stg\" (UniqueName: \"kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg\") pod \"7c5f9e33-0ade-4708-b240-676540d5322f\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") "
Jul 01 17:25:05.636368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636327    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") pod \"7c5f9e33-0ade-4708-b240-676540d5322f\" (UID: \"7c5f9e33-0ade-4708-b240-676540d5322f\") "
Jul 01 17:25:05.636565 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636460    2585 generic.go:356] "Generic (PLEG): container finished" podID="7c5f9e33-0ade-4708-b240-676540d5322f" containerID="3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1" exitCode=0
Jul 01 17:25:05.636565 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636522    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-fxn48"
Jul 01 17:25:05.636565 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636520    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-fxn48" event={"ID":"7c5f9e33-0ade-4708-b240-676540d5322f","Type":"ContainerDied","Data":"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"}
Jul 01 17:25:05.636731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636646    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-fxn48" event={"ID":"7c5f9e33-0ade-4708-b240-676540d5322f","Type":"ContainerDied","Data":"5ecfecf64e4ad6efb17a95e396cb60834ecd62d919dfff4dd52950c3fe238897"}
Jul 01 17:25:05.636731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.636684    2585 scope.go:117] "RemoveContainer" containerID="3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"
Jul 01 17:25:05.638608 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.638570    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg" (OuterVolumeSpecName: "kube-api-access-74stg") pod "7c5f9e33-0ade-4708-b240-676540d5322f" (UID: "7c5f9e33-0ade-4708-b240-676540d5322f"). InnerVolumeSpecName "kube-api-access-74stg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:25:05.638718 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.638678    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert" (OuterVolumeSpecName: "cert") pod "7c5f9e33-0ade-4708-b240-676540d5322f" (UID: "7c5f9e33-0ade-4708-b240-676540d5322f"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:25:05.648783 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.648757    2585 scope.go:117] "RemoveContainer" containerID="3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"
Jul 01 17:25:05.649031 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:25:05.649011    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1\": container with ID starting with 3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1 not found: ID does not exist" containerID="3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"
Jul 01 17:25:05.649082 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.649041    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1"} err="failed to get container status \"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1\": rpc error: code = NotFound desc = could not find container \"3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1\": container with ID starting with 3f4e1c231b327a31b6a86bcc31fc755063cc9da84775a7946c6b7c5f91f485d1 not found: ID does not exist"
Jul 01 17:25:05.692692 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.692670    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:05.737398 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.737368    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74stg\" (UniqueName: \"kubernetes.io/projected/7c5f9e33-0ade-4708-b240-676540d5322f-kube-api-access-74stg\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:25:05.737398 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.737396    2585 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c5f9e33-0ade-4708-b240-676540d5322f-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:25:05.820070 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.820035    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-848c499849-bcgpt"]
Jul 01 17:25:05.821329 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:25:05.821305    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c645c55_8652_4a51_9bc1_dc2aeeb27549.slice/crio-d3eae3436b6e1a29444b2041e671eed5148c14eb49cde95b6c4449ddab89d2b5 WatchSource:0}: Error finding container d3eae3436b6e1a29444b2041e671eed5148c14eb49cde95b6c4449ddab89d2b5: Status 404 returned error can't find the container with id d3eae3436b6e1a29444b2041e671eed5148c14eb49cde95b6c4449ddab89d2b5
Jul 01 17:25:05.957433 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.957405    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:25:05.960029 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:05.960008    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-848c499849-fxn48"]
Jul 01 17:25:06.547224 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:06.547192    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c5f9e33-0ade-4708-b240-676540d5322f" path="/var/lib/kubelet/pods/7c5f9e33-0ade-4708-b240-676540d5322f/volumes"
Jul 01 17:25:06.641990 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:06.641956    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-bcgpt" event={"ID":"0c645c55-8652-4a51-9bc1-dc2aeeb27549","Type":"ContainerStarted","Data":"d3eae3436b6e1a29444b2041e671eed5148c14eb49cde95b6c4449ddab89d2b5"}
Jul 01 17:25:07.648148 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:07.648115    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-848c499849-bcgpt" event={"ID":"0c645c55-8652-4a51-9bc1-dc2aeeb27549","Type":"ContainerStarted","Data":"68c63cec15760e718e37a58226bbb198e16ef503958766103f8cbaec96f4e24c"}
Jul 01 17:25:07.648517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:07.648298    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:07.666799 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:07.666755    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-848c499849-bcgpt" podStartSLOduration=1.619443835 podStartE2EDuration="2.666741602s" totalImagesPullingTime="1.047297767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:25:07.665033164 +0000 UTC m=+569.617459711" watchObservedRunningTime="2026-07-01 17:25:07.666741602 +0000 UTC m=+569.619168177"
Jul 01 17:25:38.503115 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:38.503085    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:25:38.503774 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:38.503450    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:25:38.512409 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:38.512385    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:25:38.513077 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:38.513060    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:25:38.657541 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:38.657514    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-848c499849-bcgpt"
Jul 01 17:25:39.594582 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.594550    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-prz8t"]
Jul 01 17:25:39.595269 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.595244    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c5f9e33-0ade-4708-b240-676540d5322f" containerName="manager"
Jul 01 17:25:39.595269 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.595270    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c5f9e33-0ade-4708-b240-676540d5322f" containerName="manager"
Jul 01 17:25:39.595448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.595353    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c5f9e33-0ade-4708-b240-676540d5322f" containerName="manager"
Jul 01 17:25:39.598264 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.598243    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:39.601417 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.601387    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\""
Jul 01 17:25:39.601796 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.601774    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-6clfz\""
Jul 01 17:25:39.607564 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.607534    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-prz8t"]
Jul 01 17:25:39.726734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.726698    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j4bq\" (UniqueName: \"kubernetes.io/projected/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-kube-api-access-5j4bq\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:39.726924 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.726772    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:39.827210 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.827172    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j4bq\" (UniqueName: \"kubernetes.io/projected/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-kube-api-access-5j4bq\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:39.827398 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.827251    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:39.827398 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:25:39.827372    2585 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 01 17:25:39.827522 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:25:39.827443    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs podName:c6a85200-d7b1-42e3-8812-ac80ff1d58d1 nodeName:}" failed. No retries permitted until 2026-07-01 17:25:40.327424086 +0000 UTC m=+602.279850625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs") pod "model-serving-api-fd65d7d6-prz8t" (UID: "c6a85200-d7b1-42e3-8812-ac80ff1d58d1") : secret "model-serving-api-tls" not found
Jul 01 17:25:39.836173 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:39.836148    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j4bq\" (UniqueName: \"kubernetes.io/projected/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-kube-api-access-5j4bq\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:40.331856 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:40.331824    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:40.334165 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:40.334141    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6a85200-d7b1-42e3-8812-ac80ff1d58d1-tls-certs\") pod \"model-serving-api-fd65d7d6-prz8t\" (UID: \"c6a85200-d7b1-42e3-8812-ac80ff1d58d1\") " pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:40.513082 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:40.513050    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:40.637065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:40.637004    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-prz8t"]
Jul 01 17:25:40.639373 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:25:40.639349    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a85200_d7b1_42e3_8812_ac80ff1d58d1.slice/crio-8256c9f689635ab5e6498f879620d1ff50ca220a3d35c1ae910e624454607dee WatchSource:0}: Error finding container 8256c9f689635ab5e6498f879620d1ff50ca220a3d35c1ae910e624454607dee: Status 404 returned error can't find the container with id 8256c9f689635ab5e6498f879620d1ff50ca220a3d35c1ae910e624454607dee
Jul 01 17:25:40.778474 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:40.778441    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-prz8t" event={"ID":"c6a85200-d7b1-42e3-8812-ac80ff1d58d1","Type":"ContainerStarted","Data":"8256c9f689635ab5e6498f879620d1ff50ca220a3d35c1ae910e624454607dee"}
Jul 01 17:25:43.794080 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:43.794046    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-prz8t" event={"ID":"c6a85200-d7b1-42e3-8812-ac80ff1d58d1","Type":"ContainerStarted","Data":"3e09637f585309e9286712da8dd95fce35f3accbd26f94903f7147ce216696fb"}
Jul 01 17:25:43.794545 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:43.794203    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:43.810165 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:43.810108    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-prz8t" podStartSLOduration=2.615219249 podStartE2EDuration="4.810090174s" totalImagesPullingTime="2.194870925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:25:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:25:43.809568781 +0000 UTC m=+605.761995330" watchObservedRunningTime="2026-07-01 17:25:43.810090174 +0000 UTC m=+605.762516723"
Jul 01 17:25:54.156821 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.156783    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-766f4878db-lz56x"]
Jul 01 17:25:54.160076 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.160045    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.170684 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.170662    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766f4878db-lz56x"]
Jul 01 17:25:54.256246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256217    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-service-ca\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256246 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256247    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256462 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256268    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-trusted-ca-bundle\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256462 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256323    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46jd\" (UniqueName: \"kubernetes.io/projected/7a781b45-8001-448c-a304-a7b9981335e3-kube-api-access-w46jd\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256462 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256370    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-oauth-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256462 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256386    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-console-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.256462 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.256422    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-oauth-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.356956 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.356914    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-service-ca\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.356969    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.356995    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-trusted-ca-bundle\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357024    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46jd\" (UniqueName: \"kubernetes.io/projected/7a781b45-8001-448c-a304-a7b9981335e3-kube-api-access-w46jd\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357051    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-oauth-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357074    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-console-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357128    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-oauth-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357828 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357798    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-service-ca\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357949 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357856    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-oauth-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.357949 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.357871    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-console-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.358063 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.358013    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a781b45-8001-448c-a304-a7b9981335e3-trusted-ca-bundle\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.359494 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.359469    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-oauth-config\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.359655 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.359638    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7a781b45-8001-448c-a304-a7b9981335e3-console-serving-cert\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.365365 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.365335    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46jd\" (UniqueName: \"kubernetes.io/projected/7a781b45-8001-448c-a304-a7b9981335e3-kube-api-access-w46jd\") pod \"console-766f4878db-lz56x\" (UID: \"7a781b45-8001-448c-a304-a7b9981335e3\") " pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.471933 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.471843    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:25:54.605584 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.605528    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-766f4878db-lz56x"]
Jul 01 17:25:54.608002 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:25:54.607976    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a781b45_8001_448c_a304_a7b9981335e3.slice/crio-cb80838d9be4a942c4fed74f517904bdeb4f7d1fbc23c6fcd4659b6722d8b7bf WatchSource:0}: Error finding container cb80838d9be4a942c4fed74f517904bdeb4f7d1fbc23c6fcd4659b6722d8b7bf: Status 404 returned error can't find the container with id cb80838d9be4a942c4fed74f517904bdeb4f7d1fbc23c6fcd4659b6722d8b7bf
Jul 01 17:25:54.803477 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.803447    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-prz8t"
Jul 01 17:25:54.842181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.842146    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766f4878db-lz56x" event={"ID":"7a781b45-8001-448c-a304-a7b9981335e3","Type":"ContainerStarted","Data":"c85bda73441625ec5a09e9ec28237c0b317ad6befa537161414a4d63e438ece3"}
Jul 01 17:25:54.842354 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.842207    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-766f4878db-lz56x" event={"ID":"7a781b45-8001-448c-a304-a7b9981335e3","Type":"ContainerStarted","Data":"cb80838d9be4a942c4fed74f517904bdeb4f7d1fbc23c6fcd4659b6722d8b7bf"}
Jul 01 17:25:54.866135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:25:54.866071    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-766f4878db-lz56x" podStartSLOduration=0.866045895 podStartE2EDuration="866.045895ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:25:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:25:54.861806899 +0000 UTC m=+616.814233445" watchObservedRunningTime="2026-07-01 17:25:54.866045895 +0000 UTC m=+616.818472443"
Jul 01 17:26:04.472072 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:04.471973    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:26:04.472072 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:04.472063    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:26:04.476838 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:04.476820    2585 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:26:04.889760 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:04.889727    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-766f4878db-lz56x"
Jul 01 17:26:04.938515 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:04.938481    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:26:29.968351 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:29.968286    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7fc6465465-2qv5q" podUID="c641ef60-3165-4542-bd02-149635f93624" containerName="console" containerID="cri-o://18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842" gracePeriod=15
Jul 01 17:26:30.217748 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.217725    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:26:30.370798 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.370769    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99vzk\" (UniqueName: \"kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.370970 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.370849    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.370970 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.370869    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.370970 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.370904    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.371135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.370970    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.371135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371002    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.371135 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371031    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle\") pod \"c641ef60-3165-4542-bd02-149635f93624\" (UID: \"c641ef60-3165-4542-bd02-149635f93624\") "
Jul 01 17:26:30.371352 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371304    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:26:30.371614 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371349    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca" (OuterVolumeSpecName: "service-ca") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:26:30.371614 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371474    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config" (OuterVolumeSpecName: "console-config") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:26:30.371614 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.371541    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:26:30.372983 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.372963    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk" (OuterVolumeSpecName: "kube-api-access-99vzk") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "kube-api-access-99vzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:26:30.373199 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.373178    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:26:30.373245 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.373203    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c641ef60-3165-4542-bd02-149635f93624" (UID: "c641ef60-3165-4542-bd02-149635f93624"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:26:30.472460 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472434    2585 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472460 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472458    2585 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c641ef60-3165-4542-bd02-149635f93624-console-oauth-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472460 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472467    2585 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-console-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472683 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472476    2585 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-trusted-ca-bundle\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472683 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472484    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99vzk\" (UniqueName: \"kubernetes.io/projected/c641ef60-3165-4542-bd02-149635f93624-kube-api-access-99vzk\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472683 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472493    2585 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-service-ca\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.472683 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.472501    2585 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c641ef60-3165-4542-bd02-149635f93624-oauth-serving-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:26:30.999558 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.999525    2585 generic.go:356] "Generic (PLEG): container finished" podID="c641ef60-3165-4542-bd02-149635f93624" containerID="18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842" exitCode=0
Jul 01 17:26:30.999982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.999581    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7fc6465465-2qv5q"
Jul 01 17:26:30.999982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.999613    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6465465-2qv5q" event={"ID":"c641ef60-3165-4542-bd02-149635f93624","Type":"ContainerDied","Data":"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"}
Jul 01 17:26:30.999982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.999652    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7fc6465465-2qv5q" event={"ID":"c641ef60-3165-4542-bd02-149635f93624","Type":"ContainerDied","Data":"b5089e786dfbb322d1d07bd5a3a82814173f84e2ad1cc469951f07f4ffbe0976"}
Jul 01 17:26:30.999982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:30.999672    2585 scope.go:117] "RemoveContainer" containerID="18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"
Jul 01 17:26:31.008357 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:31.008338    2585 scope.go:117] "RemoveContainer" containerID="18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"
Jul 01 17:26:31.008696 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:26:31.008676    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842\": container with ID starting with 18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842 not found: ID does not exist" containerID="18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"
Jul 01 17:26:31.008752 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:31.008708    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842"} err="failed to get container status \"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842\": rpc error: code = NotFound desc = could not find container \"18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842\": container with ID starting with 18be584bdaf89a5d57537a59a2c9a8608e116309f090708caad0858442b1f842 not found: ID does not exist"
Jul 01 17:26:31.016943 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:31.016921    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:26:31.020901 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:31.020878    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7fc6465465-2qv5q"]
Jul 01 17:26:32.547710 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:26:32.547678    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c641ef60-3165-4542-bd02-149635f93624" path="/var/lib/kubelet/pods/c641ef60-3165-4542-bd02-149635f93624/volumes"
Jul 01 17:27:27.070711 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.070676    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:27:27.071197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.070999    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c641ef60-3165-4542-bd02-149635f93624" containerName="console"
Jul 01 17:27:27.071197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.071011    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="c641ef60-3165-4542-bd02-149635f93624" containerName="console"
Jul 01 17:27:27.071197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.071081    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="c641ef60-3165-4542-bd02-149635f93624" containerName="console"
Jul 01 17:27:27.074324 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.074303    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.076890 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.076851    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d0081-predictor-serving-cert\""
Jul 01 17:27:27.077065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.076899    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 01 17:27:27.077065 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.077016    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-d0081-kube-rbac-proxy-sar-config\""
Jul 01 17:27:27.077198 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.077143    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 01 17:27:27.078015 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.077998    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-zg2kz\""
Jul 01 17:27:27.085984 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.085963    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:27:27.230470 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.230430    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.230651 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.230492    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.230651 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.230551    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfxvm\" (UniqueName: \"kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.332056 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.331966    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.332217 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.332051    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.332217 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.332200    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfxvm\" (UniqueName: \"kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.332822 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.332797    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.335941 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.335912    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.340702 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.340667    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfxvm\" (UniqueName: \"kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm\") pod \"success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") " pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.385549 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.385519    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:27.390671 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.390648    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:27:27.395086 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.395059    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.397695 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.397667    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 01 17:27:27.397932 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.397913    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\""
Jul 01 17:27:27.398235 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.398219    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 01 17:27:27.398304 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.398239    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\""
Jul 01 17:27:27.407674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.407108    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:27:27.529516 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.529490    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:27:27.532187 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:27:27.532158    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdcce57f_3beb_410d_84df_57c7c0b30905.slice/crio-661a5ef4ab3eb4f6e9ccc703b3ee7b2d6d48f9303683e400f40c3a66f3478212 WatchSource:0}: Error finding container 661a5ef4ab3eb4f6e9ccc703b3ee7b2d6d48f9303683e400f40c3a66f3478212: Status 404 returned error can't find the container with id 661a5ef4ab3eb4f6e9ccc703b3ee7b2d6d48f9303683e400f40c3a66f3478212
Jul 01 17:27:27.534131 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.533942    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.534131 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.534008    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h2vv\" (UniqueName: \"kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.534131 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.534091    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.534131 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.534125    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.534359 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.534159    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.534562 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.534533    2585 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 01 17:27:27.634952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.634876    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.634952 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.634927    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h2vv\" (UniqueName: \"kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.634972    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.634991    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.635013    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635707 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.635663    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635844 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.635705    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.635928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.635906    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.637681 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.637662    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.644711 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.644691    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h2vv\" (UniqueName: \"kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv\") pod \"isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.727325 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.727294    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:27.865281 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:27.865256    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:27:27.867001 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:27:27.866961    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7975c682_1ce4_4a39_8232_2db46ba88a63.slice/crio-2f8c1e285f437a576b4926886df795c3c7b16fe8c27aab57d4b71f628c638ea8 WatchSource:0}: Error finding container 2f8c1e285f437a576b4926886df795c3c7b16fe8c27aab57d4b71f628c638ea8: Status 404 returned error can't find the container with id 2f8c1e285f437a576b4926886df795c3c7b16fe8c27aab57d4b71f628c638ea8
Jul 01 17:27:28.235505 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:28.235418    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerStarted","Data":"2f8c1e285f437a576b4926886df795c3c7b16fe8c27aab57d4b71f628c638ea8"}
Jul 01 17:27:28.237051 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:28.237015    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerStarted","Data":"661a5ef4ab3eb4f6e9ccc703b3ee7b2d6d48f9303683e400f40c3a66f3478212"}
Jul 01 17:27:34.279981 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:34.279943    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerStarted","Data":"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"}
Jul 01 17:27:41.316194 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:41.316146    2585 generic.go:356] "Generic (PLEG): container finished" podID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerID="f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97" exitCode=0
Jul 01 17:27:41.316645 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:41.316255    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerDied","Data":"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"}
Jul 01 17:27:41.318187 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:41.318141    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerStarted","Data":"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"}
Jul 01 17:27:43.330081 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:43.330046    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerStarted","Data":"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"}
Jul 01 17:27:43.330529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:43.330310    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:43.330529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:43.330438    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:43.332250 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:43.332217    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:27:43.350096 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:43.350052    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podStartSLOduration=1.057307855 podStartE2EDuration="16.350038901s" totalImagesPullingTime="15.292731046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:27:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:27:43.348079671 +0000 UTC m=+725.300506217" watchObservedRunningTime="2026-07-01 17:27:43.350038901 +0000 UTC m=+725.302465445"
Jul 01 17:27:44.335372 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:44.335334    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:27:48.356823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:48.356797    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerStarted","Data":"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"}
Jul 01 17:27:49.340272 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:49.340245    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:27:49.340753 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:49.340722    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:27:49.365077 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:49.365046    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerStarted","Data":"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"}
Jul 01 17:27:49.365509 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:49.365202    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:49.391767 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:49.391724    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podStartSLOduration=5.517233364 podStartE2EDuration="22.391707757s" totalImagesPullingTime="13.072704516s" totalInitContainerRuntime="3.801769877s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:27:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:27:49.390828997 +0000 UTC m=+731.343255553" watchObservedRunningTime="2026-07-01 17:27:49.391707757 +0000 UTC m=+731.344134294"
Jul 01 17:27:50.371281 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:50.371252    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:50.372520 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:50.372490    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:27:51.376156 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:51.376116    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:27:56.380259 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:56.380229    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:27:56.380856 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:56.380829    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:27:59.341174 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:27:59.341135    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:28:06.380847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:06.380810    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:09.341653 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:09.341613    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:28:16.381234 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:16.381194    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:19.341298 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:19.341256    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.31:8080: connect: connection refused"
Jul 01 17:28:26.381821 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:26.381775    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:29.341493 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:29.341461    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:28:36.381068 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:36.381024    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:46.381586 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:46.381550    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:56.381656 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:56.381617    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:28:57.542418 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:28:57.542381    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:29:01.198422 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.198346    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:29:01.201585 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.201548    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.204265 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.204243    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6ac71-predictor-serving-cert\""
Jul 01 17:29:01.204475 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.204259    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-6ac71-kube-rbac-proxy-sar-config\""
Jul 01 17:29:01.210985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.210964    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:29:01.221633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.221611    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:29:01.221843 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.221807    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container" containerID="cri-o://c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797" gracePeriod=30
Jul 01 17:29:01.221907 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.221884    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kube-rbac-proxy" containerID="cri-o://439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3" gracePeriod=30
Jul 01 17:29:01.257404 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.257369    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.257520 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.257467    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r46qc\" (UniqueName: \"kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.257520 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.257495    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.358240 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.358210    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r46qc\" (UniqueName: \"kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.358385 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.358248    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.358385 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.358311    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.359035 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.359012    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.360642 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.360588    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.367346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.367325    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r46qc\" (UniqueName: \"kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc\") pod \"success-200-isvc-6ac71-predictor-8457887695-prfhk\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") " pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.514187 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.514154    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:01.640847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.640820    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:29:01.642474 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:29:01.642450    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33684255_0aa3_46d1_b986_1f2b9e4ed85c.slice/crio-fda3b2715152d07a858d9919e2b014dda159b528a7914c772a1dbc74604a74ef WatchSource:0}: Error finding container fda3b2715152d07a858d9919e2b014dda159b528a7914c772a1dbc74604a74ef: Status 404 returned error can't find the container with id fda3b2715152d07a858d9919e2b014dda159b528a7914c772a1dbc74604a74ef
Jul 01 17:29:01.688943 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.688918    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerStarted","Data":"fda3b2715152d07a858d9919e2b014dda159b528a7914c772a1dbc74604a74ef"}
Jul 01 17:29:01.690632 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.690573    2585 generic.go:356] "Generic (PLEG): container finished" podID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerID="439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3" exitCode=2
Jul 01 17:29:01.690716 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:01.690631    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerDied","Data":"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"}
Jul 01 17:29:02.696759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.696727    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerStarted","Data":"612f0a5c6728cbe403c4fa3578c4bb8f55bd561ca0585a57c966ada9f6d177a2"}
Jul 01 17:29:02.697197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.696783    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerStarted","Data":"aaae4c2b1c88edeb782f59fb5a77b12880291c04df14a59617e5039f533f25bb"}
Jul 01 17:29:02.697197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.696807    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:02.697197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.696941    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:02.698125 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.698105    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:02.715208 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:02.715160    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podStartSLOduration=1.715146746 podStartE2EDuration="1.715146746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:29:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:29:02.714336505 +0000 UTC m=+804.666763054" watchObservedRunningTime="2026-07-01 17:29:02.715146746 +0000 UTC m=+804.667573292"
Jul 01 17:29:03.701630 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.701573    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:03.968706 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.968684    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:29:03.980708 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.980687    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfxvm\" (UniqueName: \"kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm\") pod \"cdcce57f-3beb-410d-84df-57c7c0b30905\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") "
Jul 01 17:29:03.980794 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.980758    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls\") pod \"cdcce57f-3beb-410d-84df-57c7c0b30905\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") "
Jul 01 17:29:03.980794 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.980788    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config\") pod \"cdcce57f-3beb-410d-84df-57c7c0b30905\" (UID: \"cdcce57f-3beb-410d-84df-57c7c0b30905\") "
Jul 01 17:29:03.981173 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.981150    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "cdcce57f-3beb-410d-84df-57c7c0b30905" (UID: "cdcce57f-3beb-410d-84df-57c7c0b30905"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:29:03.982688 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.982662    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "cdcce57f-3beb-410d-84df-57c7c0b30905" (UID: "cdcce57f-3beb-410d-84df-57c7c0b30905"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:29:03.982779 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:03.982711    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm" (OuterVolumeSpecName: "kube-api-access-hfxvm") pod "cdcce57f-3beb-410d-84df-57c7c0b30905" (UID: "cdcce57f-3beb-410d-84df-57c7c0b30905"). InnerVolumeSpecName "kube-api-access-hfxvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:29:04.081834 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.081810    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfxvm\" (UniqueName: \"kubernetes.io/projected/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-api-access-hfxvm\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:04.081937 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.081830    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/cdcce57f-3beb-410d-84df-57c7c0b30905-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:04.081937 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.081854    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/cdcce57f-3beb-410d-84df-57c7c0b30905-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:04.706993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.706959    2585 generic.go:356] "Generic (PLEG): container finished" podID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerID="c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797" exitCode=0
Jul 01 17:29:04.707333 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.707030    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"
Jul 01 17:29:04.707333 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.707046    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerDied","Data":"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"}
Jul 01 17:29:04.707333 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.707143    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz" event={"ID":"cdcce57f-3beb-410d-84df-57c7c0b30905","Type":"ContainerDied","Data":"661a5ef4ab3eb4f6e9ccc703b3ee7b2d6d48f9303683e400f40c3a66f3478212"}
Jul 01 17:29:04.707333 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.707179    2585 scope.go:117] "RemoveContainer" containerID="439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"
Jul 01 17:29:04.715448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.715427    2585 scope.go:117] "RemoveContainer" containerID="c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"
Jul 01 17:29:04.724358 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.724285    2585 scope.go:117] "RemoveContainer" containerID="439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"
Jul 01 17:29:04.725017 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:29:04.724816    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3\": container with ID starting with 439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3 not found: ID does not exist" containerID="439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"
Jul 01 17:29:04.725017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.724851    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3"} err="failed to get container status \"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3\": rpc error: code = NotFound desc = could not find container \"439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3\": container with ID starting with 439de829472fd6baf0df02b5dd7de4a84a06f6c4f8938850875e0180a17726b3 not found: ID does not exist"
Jul 01 17:29:04.725017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.724873    2585 scope.go:117] "RemoveContainer" containerID="c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"
Jul 01 17:29:04.725469 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:29:04.725447    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797\": container with ID starting with c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797 not found: ID does not exist" containerID="c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"
Jul 01 17:29:04.725541 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.725480    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797"} err="failed to get container status \"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797\": rpc error: code = NotFound desc = could not find container \"c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797\": container with ID starting with c91ddd76a33c6843a97fe261db27cb50f6f767cdace0ce043aee1c60d4326797 not found: ID does not exist"
Jul 01 17:29:04.726896 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.726879    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:29:04.730029 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:04.730007    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-d0081-predictor-6b7b4d678f-x5ffz"]
Jul 01 17:29:06.547459 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:06.547422    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" path="/var/lib/kubelet/pods/cdcce57f-3beb-410d-84df-57c7c0b30905/volumes"
Jul 01 17:29:07.542921 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:07.542894    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:29:08.706300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:08.706276    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:08.706761 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:08.706736    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:18.706730 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:18.706687    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:28.706736 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:28.706699    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:38.706713 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:38.706672    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:29:47.067335 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.067304    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:29:47.067719 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.067560    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" containerID="cri-o://2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a" gracePeriod=30
Jul 01 17:29:47.067719 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.067606    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kube-rbac-proxy" containerID="cri-o://e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163" gracePeriod=30
Jul 01 17:29:47.144337 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144309    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:29:47.144809 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144787    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container"
Jul 01 17:29:47.144809 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144807    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container"
Jul 01 17:29:47.144963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144820    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kube-rbac-proxy"
Jul 01 17:29:47.144963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144826    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kube-rbac-proxy"
Jul 01 17:29:47.144963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144885    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kserve-container"
Jul 01 17:29:47.144963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.144896    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="cdcce57f-3beb-410d-84df-57c7c0b30905" containerName="kube-rbac-proxy"
Jul 01 17:29:47.147863 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.147845    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.150161 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.150142    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-766f3-predictor-serving-cert\""
Jul 01 17:29:47.150283 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.150147    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-766f3-kube-rbac-proxy-sar-config\""
Jul 01 17:29:47.159056 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.159037    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:29:47.318068 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.317991    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.318068 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.318039    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4ppq\" (UniqueName: \"kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.318232 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.318101    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.419375 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.419348    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.419529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.419395    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4ppq\" (UniqueName: \"kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.419529 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.419431    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.420161 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.420138    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.421911 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.421890    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.426934 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.426912    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4ppq\" (UniqueName: \"kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq\") pod \"success-200-isvc-766f3-predictor-788847f587-8cjdq\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") " pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.458551 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.458527    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.543149 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.543111    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 01 17:29:47.591837 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.591800    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:29:47.592440 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:29:47.592415    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaf6e27c_982c_4694_a89c_b49ac00d38e1.slice/crio-70c4f412e769e8c9bd67132b3544cff48133473612958b13dae8204434c0ba8c WatchSource:0}: Error finding container 70c4f412e769e8c9bd67132b3544cff48133473612958b13dae8204434c0ba8c: Status 404 returned error can't find the container with id 70c4f412e769e8c9bd67132b3544cff48133473612958b13dae8204434c0ba8c
Jul 01 17:29:47.899810 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.899735    2585 generic.go:356] "Generic (PLEG): container finished" podID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerID="e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163" exitCode=2
Jul 01 17:29:47.899956 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.899804    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerDied","Data":"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"}
Jul 01 17:29:47.907348 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.907307    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerStarted","Data":"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"}
Jul 01 17:29:47.907500 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.907403    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerStarted","Data":"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"}
Jul 01 17:29:47.907500 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.907437    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.907500 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.907458    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:47.907500 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.907474    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerStarted","Data":"70c4f412e769e8c9bd67132b3544cff48133473612958b13dae8204434c0ba8c"}
Jul 01 17:29:47.908918 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.908892    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:29:47.924789 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:47.924745    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podStartSLOduration=0.92473196 podStartE2EDuration="924.73196ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:29:47.92438275 +0000 UTC m=+849.876809308" watchObservedRunningTime="2026-07-01 17:29:47.92473196 +0000 UTC m=+849.877158507"
Jul 01 17:29:48.707376 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:48.707349    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:29:48.912102 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:48.912065    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:29:51.319416 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.319392    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:29:51.453061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.452987    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config\") pod \"7975c682-1ce4-4a39-8232-2db46ba88a63\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") "
Jul 01 17:29:51.453061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453019    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h2vv\" (UniqueName: \"kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv\") pod \"7975c682-1ce4-4a39-8232-2db46ba88a63\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") "
Jul 01 17:29:51.453061 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453035    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert\") pod \"7975c682-1ce4-4a39-8232-2db46ba88a63\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") "
Jul 01 17:29:51.453288 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453142    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls\") pod \"7975c682-1ce4-4a39-8232-2db46ba88a63\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") "
Jul 01 17:29:51.453288 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453195    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location\") pod \"7975c682-1ce4-4a39-8232-2db46ba88a63\" (UID: \"7975c682-1ce4-4a39-8232-2db46ba88a63\") "
Jul 01 17:29:51.453396 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453351    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "7975c682-1ce4-4a39-8232-2db46ba88a63" (UID: "7975c682-1ce4-4a39-8232-2db46ba88a63"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:29:51.453544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453515    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "7975c682-1ce4-4a39-8232-2db46ba88a63" (UID: "7975c682-1ce4-4a39-8232-2db46ba88a63"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:29:51.453633 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453575    2585 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-cabundle-cert\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:51.453678 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.453641    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "7975c682-1ce4-4a39-8232-2db46ba88a63" (UID: "7975c682-1ce4-4a39-8232-2db46ba88a63"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 01 17:29:51.455111 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.455094    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv" (OuterVolumeSpecName: "kube-api-access-5h2vv") pod "7975c682-1ce4-4a39-8232-2db46ba88a63" (UID: "7975c682-1ce4-4a39-8232-2db46ba88a63"). InnerVolumeSpecName "kube-api-access-5h2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:29:51.455163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.455145    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "7975c682-1ce4-4a39-8232-2db46ba88a63" (UID: "7975c682-1ce4-4a39-8232-2db46ba88a63"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:29:51.554504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.554472    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/7975c682-1ce4-4a39-8232-2db46ba88a63-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:51.554504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.554503    2585 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/7975c682-1ce4-4a39-8232-2db46ba88a63-kserve-provision-location\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:51.554504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.554513    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:51.554740 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.554521    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h2vv\" (UniqueName: \"kubernetes.io/projected/7975c682-1ce4-4a39-8232-2db46ba88a63-kube-api-access-5h2vv\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:29:51.927904 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.927875    2585 generic.go:356] "Generic (PLEG): container finished" podID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerID="2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a" exitCode=0
Jul 01 17:29:51.928050 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.927930    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"
Jul 01 17:29:51.928050 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.927951    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerDied","Data":"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"}
Jul 01 17:29:51.928050 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.928013    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd" event={"ID":"7975c682-1ce4-4a39-8232-2db46ba88a63","Type":"ContainerDied","Data":"2f8c1e285f437a576b4926886df795c3c7b16fe8c27aab57d4b71f628c638ea8"}
Jul 01 17:29:51.928050 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.928039    2585 scope.go:117] "RemoveContainer" containerID="e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"
Jul 01 17:29:51.936867 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.936849    2585 scope.go:117] "RemoveContainer" containerID="2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"
Jul 01 17:29:51.945818 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.945800    2585 scope.go:117] "RemoveContainer" containerID="f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"
Jul 01 17:29:51.949412 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.949391    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:29:51.954454 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954435    2585 scope.go:117] "RemoveContainer" containerID="e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"
Jul 01 17:29:51.954514 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954497    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-9c6bdd8f5-2r4nd"]
Jul 01 17:29:51.954725 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:29:51.954708    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163\": container with ID starting with e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163 not found: ID does not exist" containerID="e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"
Jul 01 17:29:51.954778 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954734    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163"} err="failed to get container status \"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163\": rpc error: code = NotFound desc = could not find container \"e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163\": container with ID starting with e1b88516c393fc4729451a51b229d7c7651649739922870a9686974a439bc163 not found: ID does not exist"
Jul 01 17:29:51.954778 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954753    2585 scope.go:117] "RemoveContainer" containerID="2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"
Jul 01 17:29:51.954980 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:29:51.954963    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a\": container with ID starting with 2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a not found: ID does not exist" containerID="2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"
Jul 01 17:29:51.955029 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954983    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a"} err="failed to get container status \"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a\": rpc error: code = NotFound desc = could not find container \"2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a\": container with ID starting with 2c05e8f9e96a3d2b1b806bcdeb2e80ff49665acec86b7308f7d14cc4201aa60a not found: ID does not exist"
Jul 01 17:29:51.955029 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.954997    2585 scope.go:117] "RemoveContainer" containerID="f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"
Jul 01 17:29:51.955191 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:29:51.955177    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97\": container with ID starting with f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97 not found: ID does not exist" containerID="f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"
Jul 01 17:29:51.955233 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:51.955193    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97"} err="failed to get container status \"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97\": rpc error: code = NotFound desc = could not find container \"f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97\": container with ID starting with f2f72c1c65d2ad4c335d86285a6046f51676b12be352d1fd88dc0a15c497bc97 not found: ID does not exist"
Jul 01 17:29:52.547778 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:52.547737    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" path="/var/lib/kubelet/pods/7975c682-1ce4-4a39-8232-2db46ba88a63/volumes"
Jul 01 17:29:53.916517 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:53.916490    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:29:53.916973 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:29:53.916947    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:30:03.917512 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:03.917475    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:30:13.917085 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:13.917033    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:30:23.917889 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:23.917849    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:30:33.917576 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:33.917497    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:30:38.532982 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:38.532956    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:30:38.533990 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:38.533967    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:30:38.539847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:38.539823    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:30:38.540537 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:30:38.540514    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:35:38.560871 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:35:38.560842    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:35:38.562491 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:35:38.562469    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:35:38.567457 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:35:38.567432    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:35:38.569016 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:35:38.568999    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:38:16.307612 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307512    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307921    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kube-rbac-proxy"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307934    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kube-rbac-proxy"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307946    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307951    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307961    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="storage-initializer"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.307967    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="storage-initializer"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.308022    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kserve-container"
Jul 01 17:38:16.308067 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.308031    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="7975c682-1ce4-4a39-8232-2db46ba88a63" containerName="kube-rbac-proxy"
Jul 01 17:38:16.311207 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.311192    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.313883 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.313854    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-43ae3-predictor-serving-cert\""
Jul 01 17:38:16.314043 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.314024    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-43ae3-kube-rbac-proxy-sar-config\""
Jul 01 17:38:16.319129 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.319111    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:38:16.319315 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.319281    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" containerID="cri-o://aaae4c2b1c88edeb782f59fb5a77b12880291c04df14a59617e5039f533f25bb" gracePeriod=30
Jul 01 17:38:16.319390 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.319362    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kube-rbac-proxy" containerID="cri-o://612f0a5c6728cbe403c4fa3578c4bb8f55bd561ca0585a57c966ada9f6d177a2" gracePeriod=30
Jul 01 17:38:16.323884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.323863    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:38:16.397682 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.397652    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f569j\" (UniqueName: \"kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.397812 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.397757    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.397812 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.397791    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.498604 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.498562    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f569j\" (UniqueName: \"kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.498733 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.498685    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.498733 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.498711    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.498902 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:38:16.498831    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-serving-cert: secret "success-200-isvc-43ae3-predictor-serving-cert" not found
Jul 01 17:38:16.498976 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:38:16.498911    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls podName:392a47f8-ca87-465e-9908-750f73ac321c nodeName:}" failed. No retries permitted until 2026-07-01 17:38:16.998893274 +0000 UTC m=+1358.951319820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls") pod "success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" (UID: "392a47f8-ca87-465e-9908-750f73ac321c") : secret "success-200-isvc-43ae3-predictor-serving-cert" not found
Jul 01 17:38:16.499504 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.499484    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:16.507185 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:16.507156    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f569j\" (UniqueName: \"kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:17.003255 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.003218    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:17.005502 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.005479    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") pod \"success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") " pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:17.174070 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.174041    2585 generic.go:356] "Generic (PLEG): container finished" podID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerID="612f0a5c6728cbe403c4fa3578c4bb8f55bd561ca0585a57c966ada9f6d177a2" exitCode=2
Jul 01 17:38:17.174197 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.174110    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerDied","Data":"612f0a5c6728cbe403c4fa3578c4bb8f55bd561ca0585a57c966ada9f6d177a2"}
Jul 01 17:38:17.223394 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.223371    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:17.366164 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:17.366035    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:38:17.368979 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:38:17.368953    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod392a47f8_ca87_465e_9908_750f73ac321c.slice/crio-3df1a98c84984f88e588b905c22458403e943cf014f68a030d4c4aa4816dfe50 WatchSource:0}: Error finding container 3df1a98c84984f88e588b905c22458403e943cf014f68a030d4c4aa4816dfe50: Status 404 returned error can't find the container with id 3df1a98c84984f88e588b905c22458403e943cf014f68a030d4c4aa4816dfe50
Jul 01 17:38:18.180875 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.180836    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerStarted","Data":"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"}
Jul 01 17:38:18.181036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.180906    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerStarted","Data":"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"}
Jul 01 17:38:18.181036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.180933    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerStarted","Data":"3df1a98c84984f88e588b905c22458403e943cf014f68a030d4c4aa4816dfe50"}
Jul 01 17:38:18.181036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.180967    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:18.205087 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.205031    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podStartSLOduration=2.2050146760000002 podStartE2EDuration="2.205014676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:38:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:38:18.197585619 +0000 UTC m=+1360.150012184" watchObservedRunningTime="2026-07-01 17:38:18.205014676 +0000 UTC m=+1360.157441222"
Jul 01 17:38:18.702437 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.702396    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 01 17:38:18.706741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:18.706716    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:8080: connect: connection refused"
Jul 01 17:38:19.188394 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.188366    2585 generic.go:356] "Generic (PLEG): container finished" podID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerID="aaae4c2b1c88edeb782f59fb5a77b12880291c04df14a59617e5039f533f25bb" exitCode=0
Jul 01 17:38:19.188523 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.188439    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerDied","Data":"aaae4c2b1c88edeb782f59fb5a77b12880291c04df14a59617e5039f533f25bb"}
Jul 01 17:38:19.188745 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.188721    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:19.189955 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.189927    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:38:19.266916 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.266892    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:38:19.318709 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.318651    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config\") pod \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") "
Jul 01 17:38:19.318823 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.318771    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r46qc\" (UniqueName: \"kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc\") pod \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") "
Jul 01 17:38:19.318872 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.318828    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls\") pod \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\" (UID: \"33684255-0aa3-46d1-b986-1f2b9e4ed85c\") "
Jul 01 17:38:19.319201 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.319003    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "33684255-0aa3-46d1-b986-1f2b9e4ed85c" (UID: "33684255-0aa3-46d1-b986-1f2b9e4ed85c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:38:19.319201 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.319114    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:38:19.320990 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.320963    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "33684255-0aa3-46d1-b986-1f2b9e4ed85c" (UID: "33684255-0aa3-46d1-b986-1f2b9e4ed85c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:38:19.321097 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.320972    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc" (OuterVolumeSpecName: "kube-api-access-r46qc") pod "33684255-0aa3-46d1-b986-1f2b9e4ed85c" (UID: "33684255-0aa3-46d1-b986-1f2b9e4ed85c"). InnerVolumeSpecName "kube-api-access-r46qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:38:19.419880 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.419854    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/33684255-0aa3-46d1-b986-1f2b9e4ed85c-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:38:19.419880 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:19.419877    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r46qc\" (UniqueName: \"kubernetes.io/projected/33684255-0aa3-46d1-b986-1f2b9e4ed85c-kube-api-access-r46qc\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:38:20.195259 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.195231    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"
Jul 01 17:38:20.195674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.195263    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk" event={"ID":"33684255-0aa3-46d1-b986-1f2b9e4ed85c","Type":"ContainerDied","Data":"fda3b2715152d07a858d9919e2b014dda159b528a7914c772a1dbc74604a74ef"}
Jul 01 17:38:20.195674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.195335    2585 scope.go:117] "RemoveContainer" containerID="612f0a5c6728cbe403c4fa3578c4bb8f55bd561ca0585a57c966ada9f6d177a2"
Jul 01 17:38:20.195674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.195620    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:38:20.205265 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.205249    2585 scope.go:117] "RemoveContainer" containerID="aaae4c2b1c88edeb782f59fb5a77b12880291c04df14a59617e5039f533f25bb"
Jul 01 17:38:20.217343 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.217320    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:38:20.220930 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.220911    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-6ac71-predictor-8457887695-prfhk"]
Jul 01 17:38:20.547484 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:20.547456    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" path="/var/lib/kubelet/pods/33684255-0aa3-46d1-b986-1f2b9e4ed85c/volumes"
Jul 01 17:38:25.200220 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:25.200188    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:38:25.200821 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:25.200792    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:38:35.201623 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:35.201564    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:38:45.201734 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:45.201695    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:38:55.200783 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:38:55.200743    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 01 17:39:01.918848 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.918818    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:39:01.919309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.919043    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" containerID="cri-o://8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c" gracePeriod=30
Jul 01 17:39:01.919309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.919073    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kube-rbac-proxy" containerID="cri-o://42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2" gracePeriod=30
Jul 01 17:39:01.959883 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.959853    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:39:01.960334 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960320    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container"
Jul 01 17:39:01.960382 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960338    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container"
Jul 01 17:39:01.960382 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960356    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kube-rbac-proxy"
Jul 01 17:39:01.960382 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960364    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kube-rbac-proxy"
Jul 01 17:39:01.960486 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960459    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kserve-container"
Jul 01 17:39:01.960486 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.960484    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="33684255-0aa3-46d1-b986-1f2b9e4ed85c" containerName="kube-rbac-proxy"
Jul 01 17:39:01.965225 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.965197    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:01.967783 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.967761    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-98f1c-kube-rbac-proxy-sar-config\""
Jul 01 17:39:01.967881 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.967834    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-98f1c-predictor-serving-cert\""
Jul 01 17:39:01.976117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:01.975898    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:39:02.125328 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.125292    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqpm4\" (UniqueName: \"kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.125472 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.125413    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.125472 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.125441    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.226008 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.225932    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqpm4\" (UniqueName: \"kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.226142 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.226034    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.226142 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.226061    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.226827 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.226804    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.228668 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.228645    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.235220 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.235197    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqpm4\" (UniqueName: \"kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4\") pod \"success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") " pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.277036 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.277012    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:02.390407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.390236    2585 generic.go:356] "Generic (PLEG): container finished" podID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerID="42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2" exitCode=2
Jul 01 17:39:02.390407 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.390338    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerDied","Data":"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"}
Jul 01 17:39:02.404655 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:02.404633    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:39:02.406189 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:39:02.406163    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42c4052a_d324_468b_9f06_7585e46b5a0e.slice/crio-6812a4afc26dcac1e029f5e23a5d17b4d5cd6bebf4b4343576b0f9080d70ccf0 WatchSource:0}: Error finding container 6812a4afc26dcac1e029f5e23a5d17b4d5cd6bebf4b4343576b0f9080d70ccf0: Status 404 returned error can't find the container with id 6812a4afc26dcac1e029f5e23a5d17b4d5cd6bebf4b4343576b0f9080d70ccf0
Jul 01 17:39:03.395946 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.395908    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerStarted","Data":"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"}
Jul 01 17:39:03.396308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.396062    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:03.396308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.396206    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:03.396308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.396247    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerStarted","Data":"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"}
Jul 01 17:39:03.396308 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.396270    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerStarted","Data":"6812a4afc26dcac1e029f5e23a5d17b4d5cd6bebf4b4343576b0f9080d70ccf0"}
Jul 01 17:39:03.397375 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.397350    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:03.413939 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.413894    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podStartSLOduration=2.413878562 podStartE2EDuration="2.413878562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:39:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:39:03.413704713 +0000 UTC m=+1405.366131288" watchObservedRunningTime="2026-07-01 17:39:03.413878562 +0000 UTC m=+1405.366305107"
Jul 01 17:39:03.912669 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.912632    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.34:8643/healthz\": dial tcp 10.134.0.34:8643: connect: connection refused"
Jul 01 17:39:03.916929 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:03.916898    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.34:8080: connect: connection refused"
Jul 01 17:39:04.400684 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.400645    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:04.768110 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.768088    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:39:04.948270 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.948208    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls\") pod \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") "
Jul 01 17:39:04.948270 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.948250    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4ppq\" (UniqueName: \"kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq\") pod \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") "
Jul 01 17:39:04.948456 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.948291    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config\") pod \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\" (UID: \"aaf6e27c-982c-4694-a89c-b49ac00d38e1\") "
Jul 01 17:39:04.948730 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.948706    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "aaf6e27c-982c-4694-a89c-b49ac00d38e1" (UID: "aaf6e27c-982c-4694-a89c-b49ac00d38e1"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:39:04.950309 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.950281    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "aaf6e27c-982c-4694-a89c-b49ac00d38e1" (UID: "aaf6e27c-982c-4694-a89c-b49ac00d38e1"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:39:04.950400 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:04.950333    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq" (OuterVolumeSpecName: "kube-api-access-r4ppq") pod "aaf6e27c-982c-4694-a89c-b49ac00d38e1" (UID: "aaf6e27c-982c-4694-a89c-b49ac00d38e1"). InnerVolumeSpecName "kube-api-access-r4ppq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:39:05.049402 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.049379    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:05.049402 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.049399    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/aaf6e27c-982c-4694-a89c-b49ac00d38e1-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:05.049520 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.049409    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4ppq\" (UniqueName: \"kubernetes.io/projected/aaf6e27c-982c-4694-a89c-b49ac00d38e1-kube-api-access-r4ppq\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:05.201381 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.201316    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:39:05.406259 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406228    2585 generic.go:356] "Generic (PLEG): container finished" podID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerID="8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c" exitCode=0
Jul 01 17:39:05.406626 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406280    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"
Jul 01 17:39:05.406626 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406317    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerDied","Data":"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"}
Jul 01 17:39:05.406626 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406380    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq" event={"ID":"aaf6e27c-982c-4694-a89c-b49ac00d38e1","Type":"ContainerDied","Data":"70c4f412e769e8c9bd67132b3544cff48133473612958b13dae8204434c0ba8c"}
Jul 01 17:39:05.406626 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406417    2585 scope.go:117] "RemoveContainer" containerID="42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"
Jul 01 17:39:05.406849 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.406797    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:05.415383 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.415369    2585 scope.go:117] "RemoveContainer" containerID="8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"
Jul 01 17:39:05.422915 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.422898    2585 scope.go:117] "RemoveContainer" containerID="42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"
Jul 01 17:39:05.423121 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:39:05.423105    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2\": container with ID starting with 42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2 not found: ID does not exist" containerID="42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"
Jul 01 17:39:05.423163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.423127    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2"} err="failed to get container status \"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2\": rpc error: code = NotFound desc = could not find container \"42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2\": container with ID starting with 42d855988aca1c3977742dea6a7dc3c9516676fe84a2034a3b5dcb4aa53b5cf2 not found: ID does not exist"
Jul 01 17:39:05.423163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.423141    2585 scope.go:117] "RemoveContainer" containerID="8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"
Jul 01 17:39:05.423346 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:39:05.423331    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c\": container with ID starting with 8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c not found: ID does not exist" containerID="8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"
Jul 01 17:39:05.423383 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.423348    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c"} err="failed to get container status \"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c\": rpc error: code = NotFound desc = could not find container \"8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c\": container with ID starting with 8c0f194afe0fc2b59f0acabe910533ebd701c6aa7506c13e84a981bb15822d9c not found: ID does not exist"
Jul 01 17:39:05.452557 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.452481    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:39:05.464784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:05.464759    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-766f3-predictor-788847f587-8cjdq"]
Jul 01 17:39:06.547940 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:06.547906    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" path="/var/lib/kubelet/pods/aaf6e27c-982c-4694-a89c-b49ac00d38e1/volumes"
Jul 01 17:39:10.411464 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:10.411430    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:10.412059 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:10.412030    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:20.412753 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:20.412711    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:26.422187 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.422149    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:39:26.422678 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.422430    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container" containerID="cri-o://2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d" gracePeriod=30
Jul 01 17:39:26.422678 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.422638    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kube-rbac-proxy" containerID="cri-o://78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc" gracePeriod=30
Jul 01 17:39:26.454686 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.454659    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:39:26.455170 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455156    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container"
Jul 01 17:39:26.455211 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455173    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container"
Jul 01 17:39:26.455211 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455184    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kube-rbac-proxy"
Jul 01 17:39:26.455211 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455192    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kube-rbac-proxy"
Jul 01 17:39:26.455301 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455256    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kube-rbac-proxy"
Jul 01 17:39:26.455301 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.455269    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaf6e27c-982c-4694-a89c-b49ac00d38e1" containerName="kserve-container"
Jul 01 17:39:26.458963 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.458945    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.461448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.461426    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c4578-predictor-serving-cert\""
Jul 01 17:39:26.461738 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.461704    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-c4578-kube-rbac-proxy-sar-config\""
Jul 01 17:39:26.467711 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.467693    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:39:26.529455 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.529423    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.529724 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.529687    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.529894 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.529760    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpkj2\" (UniqueName: \"kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.630959 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.630924    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpkj2\" (UniqueName: \"kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.631172 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.631046    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.631172 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.631078    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.631886 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.631858    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.634024 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.634004    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.647175 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.647143    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpkj2\" (UniqueName: \"kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2\") pod \"success-200-isvc-c4578-predictor-59bdd4495c-5lkdr\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") " pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.770742 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.770708    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:26.906168 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:26.906141    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:39:26.908418 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:39:26.908390    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23993103_8201_45fe_b882_e46ccab1da61.slice/crio-11a82de0299851942c823bee90328ce42a722d07e29ee09edc7ba532927f15f4 WatchSource:0}: Error finding container 11a82de0299851942c823bee90328ce42a722d07e29ee09edc7ba532927f15f4: Status 404 returned error can't find the container with id 11a82de0299851942c823bee90328ce42a722d07e29ee09edc7ba532927f15f4
Jul 01 17:39:27.515058 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.515025    2585 generic.go:356] "Generic (PLEG): container finished" podID="392a47f8-ca87-465e-9908-750f73ac321c" containerID="78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc" exitCode=2
Jul 01 17:39:27.515448 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.515100    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerDied","Data":"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"}
Jul 01 17:39:27.516876 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.516845    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerStarted","Data":"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"}
Jul 01 17:39:27.517028 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.516892    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerStarted","Data":"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"}
Jul 01 17:39:27.517028 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.516911    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerStarted","Data":"11a82de0299851942c823bee90328ce42a722d07e29ee09edc7ba532927f15f4"}
Jul 01 17:39:27.517028 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.516935    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:27.535883 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:27.535833    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podStartSLOduration=1.5358182889999998 podStartE2EDuration="1.535818289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:39:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:39:27.533909308 +0000 UTC m=+1429.486335855" watchObservedRunningTime="2026-07-01 17:39:27.535818289 +0000 UTC m=+1429.488244833"
Jul 01 17:39:28.521147 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:28.521117    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:28.522301 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:28.522276    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:39:29.525942 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:29.525851    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:39:30.077034 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.077006    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:39:30.160896 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.160801    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") pod \"392a47f8-ca87-465e-9908-750f73ac321c\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") "
Jul 01 17:39:30.160896 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.160863    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config\") pod \"392a47f8-ca87-465e-9908-750f73ac321c\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") "
Jul 01 17:39:30.160896 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.160895    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f569j\" (UniqueName: \"kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j\") pod \"392a47f8-ca87-465e-9908-750f73ac321c\" (UID: \"392a47f8-ca87-465e-9908-750f73ac321c\") "
Jul 01 17:39:30.161344 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.161305    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "392a47f8-ca87-465e-9908-750f73ac321c" (UID: "392a47f8-ca87-465e-9908-750f73ac321c"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:39:30.162998 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.162966    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "392a47f8-ca87-465e-9908-750f73ac321c" (UID: "392a47f8-ca87-465e-9908-750f73ac321c"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:39:30.163106 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.163002    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j" (OuterVolumeSpecName: "kube-api-access-f569j") pod "392a47f8-ca87-465e-9908-750f73ac321c" (UID: "392a47f8-ca87-465e-9908-750f73ac321c"). InnerVolumeSpecName "kube-api-access-f569j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:39:30.262514 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.262482    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/392a47f8-ca87-465e-9908-750f73ac321c-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:30.262514 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.262511    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/392a47f8-ca87-465e-9908-750f73ac321c-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:30.262756 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.262523    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f569j\" (UniqueName: \"kubernetes.io/projected/392a47f8-ca87-465e-9908-750f73ac321c-kube-api-access-f569j\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:39:30.412196 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.412107    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:30.532750 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.532715    2585 generic.go:356] "Generic (PLEG): container finished" podID="392a47f8-ca87-465e-9908-750f73ac321c" containerID="2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d" exitCode=0
Jul 01 17:39:30.532750 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.532753    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerDied","Data":"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"}
Jul 01 17:39:30.533206 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.532771    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"
Jul 01 17:39:30.533206 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.532805    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9" event={"ID":"392a47f8-ca87-465e-9908-750f73ac321c","Type":"ContainerDied","Data":"3df1a98c84984f88e588b905c22458403e943cf014f68a030d4c4aa4816dfe50"}
Jul 01 17:39:30.533206 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.532832    2585 scope.go:117] "RemoveContainer" containerID="78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"
Jul 01 17:39:30.541515 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.541496    2585 scope.go:117] "RemoveContainer" containerID="2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"
Jul 01 17:39:30.549213 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.549193    2585 scope.go:117] "RemoveContainer" containerID="78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"
Jul 01 17:39:30.549441 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:39:30.549424    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc\": container with ID starting with 78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc not found: ID does not exist" containerID="78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"
Jul 01 17:39:30.549487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.549450    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc"} err="failed to get container status \"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc\": rpc error: code = NotFound desc = could not find container \"78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc\": container with ID starting with 78605231f0156b8f92f68cea1ee1ad2007cacd489cf1b8c18e5aa25383c323dc not found: ID does not exist"
Jul 01 17:39:30.549487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.549467    2585 scope.go:117] "RemoveContainer" containerID="2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"
Jul 01 17:39:30.549727 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:39:30.549692    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d\": container with ID starting with 2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d not found: ID does not exist" containerID="2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"
Jul 01 17:39:30.549797 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.549737    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d"} err="failed to get container status \"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d\": rpc error: code = NotFound desc = could not find container \"2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d\": container with ID starting with 2f93d2d6534e5673e8201691e0421602713452cdcd90c9fb155db75f614cea6d not found: ID does not exist"
Jul 01 17:39:30.554258 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.554234    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:39:30.559737 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:30.559714    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-43ae3-predictor-555ccc9b67-xb6r9"]
Jul 01 17:39:32.552292 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:32.549091    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="392a47f8-ca87-465e-9908-750f73ac321c" path="/var/lib/kubelet/pods/392a47f8-ca87-465e-9908-750f73ac321c/volumes"
Jul 01 17:39:34.530714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:34.530684    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:39:34.531167 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:34.531138    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:39:40.413017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:40.412979    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 01 17:39:44.532049 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:44.532002    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:39:50.412776 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:50.412740    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:39:54.532072 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:39:54.532029    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:40:04.531809 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:04.531755    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:40:12.170857 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.170791    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:40:12.171312 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.171067    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container" containerID="cri-o://fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8" gracePeriod=30
Jul 01 17:40:12.171312 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.171139    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kube-rbac-proxy" containerID="cri-o://92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7" gracePeriod=30
Jul 01 17:40:12.192279 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192247    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:40:12.192663 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192649    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container"
Jul 01 17:40:12.192714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192664    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container"
Jul 01 17:40:12.192714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192693    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kube-rbac-proxy"
Jul 01 17:40:12.192714 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192699    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kube-rbac-proxy"
Jul 01 17:40:12.192813 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192757    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kserve-container"
Jul 01 17:40:12.192813 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.192766    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="392a47f8-ca87-465e-9908-750f73ac321c" containerName="kube-rbac-proxy"
Jul 01 17:40:12.196928 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.196469    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.199145 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.199113    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8b57f-predictor-serving-cert\""
Jul 01 17:40:12.199270 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.199141    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-8b57f-kube-rbac-proxy-sar-config\""
Jul 01 17:40:12.206921 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.206894    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:40:12.242181 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.242139    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.242694 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.242204    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.242694 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.242283    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cclhg\" (UniqueName: \"kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.343712 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.343674    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.343924 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.343729    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.343924 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.343799    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cclhg\" (UniqueName: \"kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.343924 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:40:12.343841    2585 secret.go:189] Couldn't get secret kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-serving-cert: secret "success-200-isvc-8b57f-predictor-serving-cert" not found
Jul 01 17:40:12.344064 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:40:12.343933    2585 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls podName:bb5148e0-2644-410d-b878-dc7183e9e3bb nodeName:}" failed. No retries permitted until 2026-07-01 17:40:12.843911383 +0000 UTC m=+1474.796337920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls") pod "success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" (UID: "bb5148e0-2644-410d-b878-dc7183e9e3bb") : secret "success-200-isvc-8b57f-predictor-serving-cert" not found
Jul 01 17:40:12.344465 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.344446    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.356528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.356496    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cclhg\" (UniqueName: \"kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.725510 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.725472    2585 generic.go:356] "Generic (PLEG): container finished" podID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerID="92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7" exitCode=2
Jul 01 17:40:12.725746 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.725525    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerDied","Data":"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"}
Jul 01 17:40:12.848964 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.848896    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:12.851654 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:12.851625    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") pod \"success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") " pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:13.109947 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.109888    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:13.257266 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.257235    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:40:13.259742 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:40:13.259705    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb5148e0_2644_410d_b878_dc7183e9e3bb.slice/crio-84b5fdd8b2697449ec5ff5cf5b79f1e9677f753b6292b79baac897670593b220 WatchSource:0}: Error finding container 84b5fdd8b2697449ec5ff5cf5b79f1e9677f753b6292b79baac897670593b220: Status 404 returned error can't find the container with id 84b5fdd8b2697449ec5ff5cf5b79f1e9677f753b6292b79baac897670593b220
Jul 01 17:40:13.732465 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.732366    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerStarted","Data":"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"}
Jul 01 17:40:13.732465 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.732434    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerStarted","Data":"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"}
Jul 01 17:40:13.732465 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.732458    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerStarted","Data":"84b5fdd8b2697449ec5ff5cf5b79f1e9677f753b6292b79baac897670593b220"}
Jul 01 17:40:13.732760 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.732497    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:13.732760 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.732524    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:13.734163 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.734132    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:13.751082 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:13.751017    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podStartSLOduration=1.7509942779999998 podStartE2EDuration="1.750994278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:40:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:40:13.750172344 +0000 UTC m=+1475.702598941" watchObservedRunningTime="2026-07-01 17:40:13.750994278 +0000 UTC m=+1475.703420825"
Jul 01 17:40:14.532422 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:14.532379    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:40:14.742342 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:14.742292    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:15.407096 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:15.407046    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.36:8643/healthz\": dial tcp 10.134.0.36:8643: connect: connection refused"
Jul 01 17:40:16.329168 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.329138    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:40:16.383569 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.383532    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config\") pod \"42c4052a-d324-468b-9f06-7585e46b5a0e\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") "
Jul 01 17:40:16.383769 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.383615    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls\") pod \"42c4052a-d324-468b-9f06-7585e46b5a0e\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") "
Jul 01 17:40:16.383769 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.383737    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqpm4\" (UniqueName: \"kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4\") pod \"42c4052a-d324-468b-9f06-7585e46b5a0e\" (UID: \"42c4052a-d324-468b-9f06-7585e46b5a0e\") "
Jul 01 17:40:16.383847 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.383767    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "42c4052a-d324-468b-9f06-7585e46b5a0e" (UID: "42c4052a-d324-468b-9f06-7585e46b5a0e"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:40:16.384012 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.383995    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:40:16.385794 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.385771    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4" (OuterVolumeSpecName: "kube-api-access-gqpm4") pod "42c4052a-d324-468b-9f06-7585e46b5a0e" (UID: "42c4052a-d324-468b-9f06-7585e46b5a0e"). InnerVolumeSpecName "kube-api-access-gqpm4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:40:16.385872 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.385822    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "42c4052a-d324-468b-9f06-7585e46b5a0e" (UID: "42c4052a-d324-468b-9f06-7585e46b5a0e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:40:16.484878 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.484768    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/42c4052a-d324-468b-9f06-7585e46b5a0e-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:40:16.484878 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.484817    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqpm4\" (UniqueName: \"kubernetes.io/projected/42c4052a-d324-468b-9f06-7585e46b5a0e-kube-api-access-gqpm4\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:40:16.749420 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.749386    2585 generic.go:356] "Generic (PLEG): container finished" podID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerID="fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8" exitCode=0
Jul 01 17:40:16.749625 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.749431    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerDied","Data":"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"}
Jul 01 17:40:16.749625 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.749442    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"
Jul 01 17:40:16.749625 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.749480    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs" event={"ID":"42c4052a-d324-468b-9f06-7585e46b5a0e","Type":"ContainerDied","Data":"6812a4afc26dcac1e029f5e23a5d17b4d5cd6bebf4b4343576b0f9080d70ccf0"}
Jul 01 17:40:16.749625 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.749506    2585 scope.go:117] "RemoveContainer" containerID="92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"
Jul 01 17:40:16.758090 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.758034    2585 scope.go:117] "RemoveContainer" containerID="fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"
Jul 01 17:40:16.767165 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.767141    2585 scope.go:117] "RemoveContainer" containerID="92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"
Jul 01 17:40:16.767413 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.767389    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:40:16.767523 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:40:16.767457    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7\": container with ID starting with 92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7 not found: ID does not exist" containerID="92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"
Jul 01 17:40:16.767523 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.767480    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7"} err="failed to get container status \"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7\": rpc error: code = NotFound desc = could not find container \"92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7\": container with ID starting with 92173fbd021e33b21b2a967461c54ecf8baf7c157d34566510672f93a21ba8f7 not found: ID does not exist"
Jul 01 17:40:16.767523 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.767500    2585 scope.go:117] "RemoveContainer" containerID="fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"
Jul 01 17:40:16.767814 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:40:16.767796    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8\": container with ID starting with fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8 not found: ID does not exist" containerID="fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"
Jul 01 17:40:16.767860 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.767820    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8"} err="failed to get container status \"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8\": rpc error: code = NotFound desc = could not find container \"fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8\": container with ID starting with fbabfc5ca33aef10a42cab4e743ce4c67858f6724eb98835186ce19b1ecfaff8 not found: ID does not exist"
Jul 01 17:40:16.770011 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:16.769991    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-98f1c-predictor-659b84d7f7-f2vvs"]
Jul 01 17:40:18.547239 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:18.547204    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" path="/var/lib/kubelet/pods/42c4052a-d324-468b-9f06-7585e46b5a0e/volumes"
Jul 01 17:40:19.744319 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:19.744289    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:40:19.744803 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:19.744775    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:29.744851 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:29.744807    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:38.588777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:38.588744    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:40:38.591346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:38.591323    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:40:38.595833 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:38.595809    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:40:38.598199 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:38.598177    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:40:39.745533 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:39.745490    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:49.745758 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:49.745710    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:40:59.745193 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:40:59.745104    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:41:09.745788 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:41:09.745753    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:45:38.618684 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:45:38.618654    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:45:38.621152 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:45:38.621123    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:45:38.625330 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:45:38.625307    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:45:38.627880 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:45:38.627863    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:48:41.435222 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.435185    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:48:41.435827 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.435448    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" containerID="cri-o://354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796" gracePeriod=30
Jul 01 17:48:41.435827 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.435641    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kube-rbac-proxy" containerID="cri-o://f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed" gracePeriod=30
Jul 01 17:48:41.438985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.438961    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:48:41.439544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439520    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kube-rbac-proxy"
Jul 01 17:48:41.439544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439541    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kube-rbac-proxy"
Jul 01 17:48:41.439723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439567    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container"
Jul 01 17:48:41.439723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439572    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container"
Jul 01 17:48:41.439723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439677    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kube-rbac-proxy"
Jul 01 17:48:41.439723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.439688    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="42c4052a-d324-468b-9f06-7585e46b5a0e" containerName="kserve-container"
Jul 01 17:48:41.443174 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.443155    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.445353 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.445327    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-10517-predictor-serving-cert\""
Jul 01 17:48:41.445452 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.445400    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-10517-kube-rbac-proxy-sar-config\""
Jul 01 17:48:41.455062 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.455010    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:48:41.578001 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.577968    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgqv9\" (UniqueName: \"kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.578157 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.578100    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.578157 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.578134    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.679674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.679633    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.679674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.679674    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.679974 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.679713    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgqv9\" (UniqueName: \"kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.680518 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.680488    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.682253 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.682231    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.687791 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.687738    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgqv9\" (UniqueName: \"kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9\") pod \"success-200-isvc-10517-predictor-54485f9447-6xsj6\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") " pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.764316 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.764280    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:41.900691 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:41.900666    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:48:42.024658 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:42.024588    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerStarted","Data":"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"}
Jul 01 17:48:42.024738 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:42.024664    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerStarted","Data":"a99932ce91e428f6eeb0a597da1614e5ff80b02b936e8e0e8cb5bcfcf2d0b79f"}
Jul 01 17:48:42.026267 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:42.026242    2585 generic.go:356] "Generic (PLEG): container finished" podID="23993103-8201-45fe-b882-e46ccab1da61" containerID="f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed" exitCode=2
Jul 01 17:48:42.026350 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:42.026277    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerDied","Data":"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"}
Jul 01 17:48:43.032167 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:43.032132    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerStarted","Data":"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"}
Jul 01 17:48:43.038759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:43.032768    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:43.038759 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:43.033067    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:43.039009 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:43.038875    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:48:43.053888 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:43.053833    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podStartSLOduration=2.0538164 podStartE2EDuration="2.0538164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:48:43.052062687 +0000 UTC m=+1985.004489236" watchObservedRunningTime="2026-07-01 17:48:43.0538164 +0000 UTC m=+1985.006242946"
Jul 01 17:48:44.037086 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.037042    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:48:44.526142 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.526104    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.37:8643/healthz\": dial tcp 10.134.0.37:8643: connect: connection refused"
Jul 01 17:48:44.531085 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.531063    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.37:8080: connect: connection refused"
Jul 01 17:48:44.894838 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.894814    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:48:44.913044 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.913013    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls\") pod \"23993103-8201-45fe-b882-e46ccab1da61\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") "
Jul 01 17:48:44.913199 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.913064    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpkj2\" (UniqueName: \"kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2\") pod \"23993103-8201-45fe-b882-e46ccab1da61\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") "
Jul 01 17:48:44.913199 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.913143    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config\") pod \"23993103-8201-45fe-b882-e46ccab1da61\" (UID: \"23993103-8201-45fe-b882-e46ccab1da61\") "
Jul 01 17:48:44.913550 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.913521    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "23993103-8201-45fe-b882-e46ccab1da61" (UID: "23993103-8201-45fe-b882-e46ccab1da61"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:48:44.915368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.915336    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2" (OuterVolumeSpecName: "kube-api-access-qpkj2") pod "23993103-8201-45fe-b882-e46ccab1da61" (UID: "23993103-8201-45fe-b882-e46ccab1da61"). InnerVolumeSpecName "kube-api-access-qpkj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:48:44.915368 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:44.915349    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "23993103-8201-45fe-b882-e46ccab1da61" (UID: "23993103-8201-45fe-b882-e46ccab1da61"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:48:45.013993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.013959    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/23993103-8201-45fe-b882-e46ccab1da61-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:48:45.013993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.013988    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpkj2\" (UniqueName: \"kubernetes.io/projected/23993103-8201-45fe-b882-e46ccab1da61-kube-api-access-qpkj2\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:48:45.013993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.013999    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/23993103-8201-45fe-b882-e46ccab1da61-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:48:45.042365 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.042328    2585 generic.go:356] "Generic (PLEG): container finished" podID="23993103-8201-45fe-b882-e46ccab1da61" containerID="354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796" exitCode=0
Jul 01 17:48:45.042741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.042387    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"
Jul 01 17:48:45.042741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.042406    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerDied","Data":"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"}
Jul 01 17:48:45.042741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.042472    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr" event={"ID":"23993103-8201-45fe-b882-e46ccab1da61","Type":"ContainerDied","Data":"11a82de0299851942c823bee90328ce42a722d07e29ee09edc7ba532927f15f4"}
Jul 01 17:48:45.042741 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.042499    2585 scope.go:117] "RemoveContainer" containerID="f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"
Jul 01 17:48:45.051895 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.051878    2585 scope.go:117] "RemoveContainer" containerID="354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"
Jul 01 17:48:45.059856 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.059833    2585 scope.go:117] "RemoveContainer" containerID="f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"
Jul 01 17:48:45.060122 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:48:45.060102    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed\": container with ID starting with f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed not found: ID does not exist" containerID="f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"
Jul 01 17:48:45.060226 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.060132    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed"} err="failed to get container status \"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed\": rpc error: code = NotFound desc = could not find container \"f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed\": container with ID starting with f4faf88db581f9682fd6dc399076b6a29683f4eecccb50a53be25f47e07b13ed not found: ID does not exist"
Jul 01 17:48:45.060226 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.060149    2585 scope.go:117] "RemoveContainer" containerID="354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"
Jul 01 17:48:45.060426 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:48:45.060406    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796\": container with ID starting with 354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796 not found: ID does not exist" containerID="354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"
Jul 01 17:48:45.060501 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.060437    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796"} err="failed to get container status \"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796\": rpc error: code = NotFound desc = could not find container \"354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796\": container with ID starting with 354d383bc50f8b8e0d92dbbe707c72c91436222c0e86f0ad7921e8213b8ee796 not found: ID does not exist"
Jul 01 17:48:45.065873 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.065847    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:48:45.071141 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:45.071118    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-c4578-predictor-59bdd4495c-5lkdr"]
Jul 01 17:48:46.547218 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:46.547185    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23993103-8201-45fe-b882-e46ccab1da61" path="/var/lib/kubelet/pods/23993103-8201-45fe-b882-e46ccab1da61/volumes"
Jul 01 17:48:49.042346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:49.042318    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:48:49.042875 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:49.042845    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:48:59.043737 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:48:59.043698    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:49:09.043072 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:09.043035    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:49:19.043436 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:19.043401    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.39:8080: connect: connection refused"
Jul 01 17:49:29.043747 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:29.043714    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:49:37.054205 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.054164    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:49:37.054720 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.054440    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" containerID="cri-o://71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9" gracePeriod=30
Jul 01 17:49:37.054720 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.054434    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kube-rbac-proxy" containerID="cri-o://22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41" gracePeriod=30
Jul 01 17:49:37.106526 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.106489    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 17:49:37.106994 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.106973    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container"
Jul 01 17:49:37.106994 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.106995    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container"
Jul 01 17:49:37.107200 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.107029    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kube-rbac-proxy"
Jul 01 17:49:37.107200 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.107037    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kube-rbac-proxy"
Jul 01 17:49:37.107200 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.107128    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kserve-container"
Jul 01 17:49:37.107200 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.107143    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="23993103-8201-45fe-b882-e46ccab1da61" containerName="kube-rbac-proxy"
Jul 01 17:49:37.110640 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.110615    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.113214 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.113190    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4822a-kube-rbac-proxy-sar-config\""
Jul 01 17:49:37.113349 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.113193    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-4822a-predictor-serving-cert\""
Jul 01 17:49:37.121033 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.121008    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 17:49:37.251512 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.251457    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.251723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.251537    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.251723 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.251569    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvjf8\" (UniqueName: \"kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.281674 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.281636    2585 generic.go:356] "Generic (PLEG): container finished" podID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerID="22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41" exitCode=2
Jul 01 17:49:37.281845 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.281706    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerDied","Data":"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"}
Jul 01 17:49:37.352993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.352896    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.352993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.352958    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.352993 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.352979    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvjf8\" (UniqueName: \"kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.353985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.353953    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.355622 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.355581    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.360950 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.360922    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvjf8\" (UniqueName: \"kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8\") pod \"success-200-isvc-4822a-predictor-59f89c648f-gb6hs\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") " pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.423586 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.423545    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:37.557298 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:37.557273    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 17:49:37.559230 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:49:37.559200    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d44ed3_a82c_4902_8efc_b6d41630893a.slice/crio-1a39a3f60bd2b3687ed574178c2c6d5b12ecdc2402d164583207d3598de1c868 WatchSource:0}: Error finding container 1a39a3f60bd2b3687ed574178c2c6d5b12ecdc2402d164583207d3598de1c868: Status 404 returned error can't find the container with id 1a39a3f60bd2b3687ed574178c2c6d5b12ecdc2402d164583207d3598de1c868
Jul 01 17:49:38.289701 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:38.289654    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerStarted","Data":"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"}
Jul 01 17:49:38.290111 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:38.289734    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerStarted","Data":"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"}
Jul 01 17:49:38.290111 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:38.289764    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerStarted","Data":"1a39a3f60bd2b3687ed574178c2c6d5b12ecdc2402d164583207d3598de1c868"}
Jul 01 17:49:38.290111 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:38.289917    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:38.311704 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:38.311658    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podStartSLOduration=1.311643892 podStartE2EDuration="1.311643892s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:49:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:49:38.309755341 +0000 UTC m=+2040.262181926" watchObservedRunningTime="2026-07-01 17:49:38.311643892 +0000 UTC m=+2040.264070437"
Jul 01 17:49:39.294450 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:39.294417    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:39.295848 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:39.295820    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:49:39.740941 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:39.740904    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Jul 01 17:49:39.745260 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:39.745239    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 01 17:49:40.215208 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.215186    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:49:40.300239 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300208    2585 generic.go:356] "Generic (PLEG): container finished" podID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerID="71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9" exitCode=0
Jul 01 17:49:40.300664 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300261    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"
Jul 01 17:49:40.300664 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300287    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerDied","Data":"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"}
Jul 01 17:49:40.300664 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300341    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq" event={"ID":"bb5148e0-2644-410d-b878-dc7183e9e3bb","Type":"ContainerDied","Data":"84b5fdd8b2697449ec5ff5cf5b79f1e9677f753b6292b79baac897670593b220"}
Jul 01 17:49:40.300664 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300367    2585 scope.go:117] "RemoveContainer" containerID="22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"
Jul 01 17:49:40.300882 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.300817    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:49:40.308662 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.308644    2585 scope.go:117] "RemoveContainer" containerID="71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"
Jul 01 17:49:40.316037 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.316019    2585 scope.go:117] "RemoveContainer" containerID="22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"
Jul 01 17:49:40.316277 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:49:40.316260    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41\": container with ID starting with 22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41 not found: ID does not exist" containerID="22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"
Jul 01 17:49:40.316346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.316289    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41"} err="failed to get container status \"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41\": rpc error: code = NotFound desc = could not find container \"22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41\": container with ID starting with 22c32fc4e3c982677037f9a782ac1b21e52a3439c10e887d33c5e7a4d8a53d41 not found: ID does not exist"
Jul 01 17:49:40.316346 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.316311    2585 scope.go:117] "RemoveContainer" containerID="71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"
Jul 01 17:49:40.316542 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:49:40.316525    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9\": container with ID starting with 71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9 not found: ID does not exist" containerID="71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"
Jul 01 17:49:40.316604 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.316548    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9"} err="failed to get container status \"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9\": rpc error: code = NotFound desc = could not find container \"71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9\": container with ID starting with 71a805e6c3829c08fd70e4c9f5616662b7c816bda0c3a4265cdc584a9a2a9de9 not found: ID does not exist"
Jul 01 17:49:40.382964 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.382933    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cclhg\" (UniqueName: \"kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg\") pod \"bb5148e0-2644-410d-b878-dc7183e9e3bb\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") "
Jul 01 17:49:40.383073 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.382986    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config\") pod \"bb5148e0-2644-410d-b878-dc7183e9e3bb\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") "
Jul 01 17:49:40.383073 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.383027    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") pod \"bb5148e0-2644-410d-b878-dc7183e9e3bb\" (UID: \"bb5148e0-2644-410d-b878-dc7183e9e3bb\") "
Jul 01 17:49:40.383337 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.383313    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "bb5148e0-2644-410d-b878-dc7183e9e3bb" (UID: "bb5148e0-2644-410d-b878-dc7183e9e3bb"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:49:40.385017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.384986    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg" (OuterVolumeSpecName: "kube-api-access-cclhg") pod "bb5148e0-2644-410d-b878-dc7183e9e3bb" (UID: "bb5148e0-2644-410d-b878-dc7183e9e3bb"). InnerVolumeSpecName "kube-api-access-cclhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:49:40.385017 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.385003    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "bb5148e0-2644-410d-b878-dc7183e9e3bb" (UID: "bb5148e0-2644-410d-b878-dc7183e9e3bb"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:49:40.483871 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.483833    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cclhg\" (UniqueName: \"kubernetes.io/projected/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-api-access-cclhg\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:40.483871 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.483870    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/bb5148e0-2644-410d-b878-dc7183e9e3bb-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:40.484012 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.483885    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/bb5148e0-2644-410d-b878-dc7183e9e3bb-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:40.616351 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.616284    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:49:40.620072 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:40.620047    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-8b57f-predictor-68d7874c4c-h6pdq"]
Jul 01 17:49:42.546860 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:42.546823    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" path="/var/lib/kubelet/pods/bb5148e0-2644-410d-b878-dc7183e9e3bb/volumes"
Jul 01 17:49:45.305264 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:45.305228    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:49:45.305840 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:45.305813    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:49:51.641366 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.641335    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:49:51.641985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.641638    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container" containerID="cri-o://a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b" gracePeriod=30
Jul 01 17:49:51.641985 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.641884    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kube-rbac-proxy" containerID="cri-o://31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7" gracePeriod=30
Jul 01 17:49:51.680383 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680346    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:49:51.680792 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680775    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kube-rbac-proxy"
Jul 01 17:49:51.680884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680795    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kube-rbac-proxy"
Jul 01 17:49:51.680884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680809    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container"
Jul 01 17:49:51.680884 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680817    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container"
Jul 01 17:49:51.681113 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680901    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kserve-container"
Jul 01 17:49:51.681113 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.680920    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb5148e0-2644-410d-b878-dc7183e9e3bb" containerName="kube-rbac-proxy"
Jul 01 17:49:51.685487 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.685467    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.687731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.687712    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-42b28-predictor-serving-cert\""
Jul 01 17:49:51.687731 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.687722    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"success-200-isvc-42b28-kube-rbac-proxy-sar-config\""
Jul 01 17:49:51.692489 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.692465    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:49:51.778834 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.778795    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4fbb\" (UniqueName: \"kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.778834 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.778836    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.779032 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.778863    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.879876 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.879840    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4fbb\" (UniqueName: \"kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.879876 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.879878    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.880081 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.879910    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.880675 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.880653    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.882572 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.882548    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.887605 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.887567    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4fbb\" (UniqueName: \"kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb\") pod \"success-200-isvc-42b28-predictor-596b5858f9-q5wps\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") " pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:51.997544 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:51.997516    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:52.129543 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.129517    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:49:52.131578 ip-10-0-139-93 kubenswrapper[2585]: W0701 17:49:52.131547    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17084014_45f0_4920_bfc3_99d6187448ff.slice/crio-bf34213f6e45b1c20647f897661cf88b35a75426f16c8c96a88524eb7243c970 WatchSource:0}: Error finding container bf34213f6e45b1c20647f897661cf88b35a75426f16c8c96a88524eb7243c970: Status 404 returned error can't find the container with id bf34213f6e45b1c20647f897661cf88b35a75426f16c8c96a88524eb7243c970
Jul 01 17:49:52.358577 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.358539    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerStarted","Data":"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"}
Jul 01 17:49:52.358784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.358633    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerStarted","Data":"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"}
Jul 01 17:49:52.358784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.358671    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:52.358784 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.358700    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerStarted","Data":"bf34213f6e45b1c20647f897661cf88b35a75426f16c8c96a88524eb7243c970"}
Jul 01 17:49:52.360777 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.360746    2585 generic.go:356] "Generic (PLEG): container finished" podID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerID="31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7" exitCode=2
Jul 01 17:49:52.360920 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.360817    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerDied","Data":"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"}
Jul 01 17:49:52.375833 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:52.375784    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podStartSLOduration=1.375768005 podStartE2EDuration="1.375768005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:49:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:49:52.37482289 +0000 UTC m=+2054.327249436" watchObservedRunningTime="2026-07-01 17:49:52.375768005 +0000 UTC m=+2054.328194549"
Jul 01 17:49:53.367145 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:53.367099    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:53.368771 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:53.368740    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:49:54.037385 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:54.037343    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.39:8643/healthz\": dial tcp 10.134.0.39:8643: connect: connection refused"
Jul 01 17:49:54.371807 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:54.371709    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:49:54.896221 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:54.896193    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:49:55.006191 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.006163    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config\") pod \"4d77ffc0-8d6c-48f1-90ae-abe293506504\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") "
Jul 01 17:49:55.006336 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.006231    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgqv9\" (UniqueName: \"kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9\") pod \"4d77ffc0-8d6c-48f1-90ae-abe293506504\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") "
Jul 01 17:49:55.006336 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.006275    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls\") pod \"4d77ffc0-8d6c-48f1-90ae-abe293506504\" (UID: \"4d77ffc0-8d6c-48f1-90ae-abe293506504\") "
Jul 01 17:49:55.006551 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.006522    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "4d77ffc0-8d6c-48f1-90ae-abe293506504" (UID: "4d77ffc0-8d6c-48f1-90ae-abe293506504"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:49:55.008233 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.008213    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9" (OuterVolumeSpecName: "kube-api-access-bgqv9") pod "4d77ffc0-8d6c-48f1-90ae-abe293506504" (UID: "4d77ffc0-8d6c-48f1-90ae-abe293506504"). InnerVolumeSpecName "kube-api-access-bgqv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:49:55.008300 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.008285    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "4d77ffc0-8d6c-48f1-90ae-abe293506504" (UID: "4d77ffc0-8d6c-48f1-90ae-abe293506504"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:49:55.106946 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.106917    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:55.106946 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.106942    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgqv9\" (UniqueName: \"kubernetes.io/projected/4d77ffc0-8d6c-48f1-90ae-abe293506504-kube-api-access-bgqv9\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:55.107099 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.106952    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/4d77ffc0-8d6c-48f1-90ae-abe293506504-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:49:55.306244 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.306163    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:49:55.376623 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.376576    2585 generic.go:356] "Generic (PLEG): container finished" podID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerID="a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b" exitCode=0
Jul 01 17:49:55.376980 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.376642    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"
Jul 01 17:49:55.376980 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.376664    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerDied","Data":"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"}
Jul 01 17:49:55.376980 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.376725    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6" event={"ID":"4d77ffc0-8d6c-48f1-90ae-abe293506504","Type":"ContainerDied","Data":"a99932ce91e428f6eeb0a597da1614e5ff80b02b936e8e0e8cb5bcfcf2d0b79f"}
Jul 01 17:49:55.376980 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.376752    2585 scope.go:117] "RemoveContainer" containerID="31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"
Jul 01 17:49:55.385206 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.385190    2585 scope.go:117] "RemoveContainer" containerID="a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"
Jul 01 17:49:55.392430 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.392413    2585 scope.go:117] "RemoveContainer" containerID="31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"
Jul 01 17:49:55.392689 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:49:55.392671    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7\": container with ID starting with 31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7 not found: ID does not exist" containerID="31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"
Jul 01 17:49:55.392751 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.392696    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7"} err="failed to get container status \"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7\": rpc error: code = NotFound desc = could not find container \"31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7\": container with ID starting with 31a248d3907cbf752bad9f5476f87c9a6b5adb4703c6583f1497ac41e65a18a7 not found: ID does not exist"
Jul 01 17:49:55.392751 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.392712    2585 scope.go:117] "RemoveContainer" containerID="a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"
Jul 01 17:49:55.392972 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:49:55.392954    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b\": container with ID starting with a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b not found: ID does not exist" containerID="a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"
Jul 01 17:49:55.393025 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.392981    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b"} err="failed to get container status \"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b\": rpc error: code = NotFound desc = could not find container \"a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b\": container with ID starting with a3ea6885895009f1eb020b2a89676b0cb33dc4dc0db8ce6409ce099384a8671b not found: ID does not exist"
Jul 01 17:49:55.403150 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.403129    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:49:55.427389 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:55.427368    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-10517-predictor-54485f9447-6xsj6"]
Jul 01 17:49:56.547195 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:56.547158    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" path="/var/lib/kubelet/pods/4d77ffc0-8d6c-48f1-90ae-abe293506504/volumes"
Jul 01 17:49:59.376588 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:59.376558    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:49:59.377094 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:49:59.377065    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:50:05.306253 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:05.306214    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:50:09.377140 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:09.377096    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:50:15.305834 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:15.305795    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 01 17:50:19.377978 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:19.377941    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:50:25.306536 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:25.306506    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 17:50:29.378018 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:29.377972    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:50:38.646699 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:38.646666    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:50:38.650895 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:38.650867    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:50:38.654155 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:38.654131    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:50:38.658313 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:38.658295    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:50:39.378225 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:50:39.378197    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:55:38.674396 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:55:38.674363    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:55:38.680296 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:55:38.680274    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 17:55:38.680897 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:55:38.680877    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:55:38.686454 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:55:38.686438    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 17:59:06.364081 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:06.363989    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:59:06.364728 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:06.364262    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" containerID="cri-o://26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93" gracePeriod=30
Jul 01 17:59:06.364728 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:06.364301    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kube-rbac-proxy" containerID="cri-o://7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943" gracePeriod=30
Jul 01 17:59:06.853193 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:06.853153    2585 generic.go:356] "Generic (PLEG): container finished" podID="17084014-45f0-4920-bfc3-99d6187448ff" containerID="7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943" exitCode=2
Jul 01 17:59:06.853402 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:06.853227    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerDied","Data":"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"}
Jul 01 17:59:09.371973 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:09.371922    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.41:8643/healthz\": dial tcp 10.134.0.41:8643: connect: connection refused"
Jul 01 17:59:09.377283 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:09.377256    2585 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.41:8080: connect: connection refused"
Jul 01 17:59:14.034229 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.034202    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:59:14.058584 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.058547    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4fbb\" (UniqueName: \"kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb\") pod \"17084014-45f0-4920-bfc3-99d6187448ff\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") "
Jul 01 17:59:14.058767 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.058640    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls\") pod \"17084014-45f0-4920-bfc3-99d6187448ff\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") "
Jul 01 17:59:14.058767 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.058682    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config\") pod \"17084014-45f0-4920-bfc3-99d6187448ff\" (UID: \"17084014-45f0-4920-bfc3-99d6187448ff\") "
Jul 01 17:59:14.059117 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.059080    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "17084014-45f0-4920-bfc3-99d6187448ff" (UID: "17084014-45f0-4920-bfc3-99d6187448ff"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 17:59:14.061134 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.061106    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "17084014-45f0-4920-bfc3-99d6187448ff" (UID: "17084014-45f0-4920-bfc3-99d6187448ff"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 17:59:14.061262 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.061129    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb" (OuterVolumeSpecName: "kube-api-access-r4fbb") pod "17084014-45f0-4920-bfc3-99d6187448ff" (UID: "17084014-45f0-4920-bfc3-99d6187448ff"). InnerVolumeSpecName "kube-api-access-r4fbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 17:59:14.159528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.159429    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4fbb\" (UniqueName: \"kubernetes.io/projected/17084014-45f0-4920-bfc3-99d6187448ff-kube-api-access-r4fbb\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:59:14.159528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.159462    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/17084014-45f0-4920-bfc3-99d6187448ff-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:59:14.159528 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.159473    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/17084014-45f0-4920-bfc3-99d6187448ff-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 17:59:14.891622 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.891519    2585 generic.go:356] "Generic (PLEG): container finished" podID="17084014-45f0-4920-bfc3-99d6187448ff" containerID="26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93" exitCode=0
Jul 01 17:59:14.891622 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.891585    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"
Jul 01 17:59:14.891622 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.891609    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerDied","Data":"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"}
Jul 01 17:59:14.891854 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.891712    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps" event={"ID":"17084014-45f0-4920-bfc3-99d6187448ff","Type":"ContainerDied","Data":"bf34213f6e45b1c20647f897661cf88b35a75426f16c8c96a88524eb7243c970"}
Jul 01 17:59:14.891854 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.891754    2585 scope.go:117] "RemoveContainer" containerID="7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"
Jul 01 17:59:14.902933 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.902908    2585 scope.go:117] "RemoveContainer" containerID="26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"
Jul 01 17:59:14.910145 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.910119    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:59:14.912129 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.912099    2585 scope.go:117] "RemoveContainer" containerID="7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"
Jul 01 17:59:14.912442 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:59:14.912422    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943\": container with ID starting with 7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943 not found: ID does not exist" containerID="7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"
Jul 01 17:59:14.912518 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.912453    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943"} err="failed to get container status \"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943\": rpc error: code = NotFound desc = could not find container \"7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943\": container with ID starting with 7bd249d9e7ea1add0f54786c08b1057978079a0fbdaaac9386bb8cebede8e943 not found: ID does not exist"
Jul 01 17:59:14.912518 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.912475    2585 scope.go:117] "RemoveContainer" containerID="26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"
Jul 01 17:59:14.912797 ip-10-0-139-93 kubenswrapper[2585]: E0701 17:59:14.912779    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93\": container with ID starting with 26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93 not found: ID does not exist" containerID="26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"
Jul 01 17:59:14.912852 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.912805    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93"} err="failed to get container status \"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93\": rpc error: code = NotFound desc = could not find container \"26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93\": container with ID starting with 26c28cf2232e89abe9d0ce7253c2f16fa7f878e7ed8d7f483beb4e022dd5bb93 not found: ID does not exist"
Jul 01 17:59:14.913786 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:14.913768    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-42b28-predictor-596b5858f9-q5wps"]
Jul 01 17:59:16.547296 ip-10-0-139-93 kubenswrapper[2585]: I0701 17:59:16.547259    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17084014-45f0-4920-bfc3-99d6187448ff" path="/var/lib/kubelet/pods/17084014-45f0-4920-bfc3-99d6187448ff/volumes"
Jul 01 18:00:38.700857 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:00:38.700747    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 18:00:38.706962 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:00:38.706942    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 18:00:38.707652 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:00:38.707630    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 18:00:38.713409 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:00:38.713393    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 18:05:38.730027 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:05:38.729903    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 18:05:38.744394 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:05:38.744360    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 18:05:38.744910 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:05:38.744892    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 18:05:38.754587 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:05:38.754571    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 18:06:56.602367 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:56.602337    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 18:06:56.602931 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:56.602564    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container" containerID="cri-o://567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390" gracePeriod=30
Jul 01 18:06:56.602931 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:56.602611    2585 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kube-rbac-proxy" containerID="cri-o://90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01" gracePeriod=30
Jul 01 18:06:56.844326 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:56.844293    2585 generic.go:356] "Generic (PLEG): container finished" podID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerID="90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01" exitCode=2
Jul 01 18:06:56.844443 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:56.844369    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerDied","Data":"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"}
Jul 01 18:06:59.149094 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.149072    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 18:06:59.295943 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.295920    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvjf8\" (UniqueName: \"kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8\") pod \"f4d44ed3-a82c-4902-8efc-b6d41630893a\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") "
Jul 01 18:06:59.296105 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.295957    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls\") pod \"f4d44ed3-a82c-4902-8efc-b6d41630893a\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") "
Jul 01 18:06:59.296105 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.295987    2585 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config\") pod \"f4d44ed3-a82c-4902-8efc-b6d41630893a\" (UID: \"f4d44ed3-a82c-4902-8efc-b6d41630893a\") "
Jul 01 18:06:59.296375 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.296343    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "f4d44ed3-a82c-4902-8efc-b6d41630893a" (UID: "f4d44ed3-a82c-4902-8efc-b6d41630893a"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 01 18:06:59.298036 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.298010    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8" (OuterVolumeSpecName: "kube-api-access-bvjf8") pod "f4d44ed3-a82c-4902-8efc-b6d41630893a" (UID: "f4d44ed3-a82c-4902-8efc-b6d41630893a"). InnerVolumeSpecName "kube-api-access-bvjf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 01 18:06:59.298122 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.298032    2585 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "f4d44ed3-a82c-4902-8efc-b6d41630893a" (UID: "f4d44ed3-a82c-4902-8efc-b6d41630893a"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 01 18:06:59.396735 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.396712    2585 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/f4d44ed3-a82c-4902-8efc-b6d41630893a-proxy-tls\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 18:06:59.396735 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.396732    2585 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-rbac-proxy-sar-config\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 18:06:59.396864 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.396740    2585 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvjf8\" (UniqueName: \"kubernetes.io/projected/f4d44ed3-a82c-4902-8efc-b6d41630893a-kube-api-access-bvjf8\") on node \"ip-10-0-139-93.ec2.internal\" DevicePath \"\""
Jul 01 18:06:59.857813 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.857776    2585 generic.go:356] "Generic (PLEG): container finished" podID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerID="567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390" exitCode=0
Jul 01 18:06:59.857998 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.857824    2585 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"
Jul 01 18:06:59.857998 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.857867    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerDied","Data":"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"}
Jul 01 18:06:59.857998 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.857941    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs" event={"ID":"f4d44ed3-a82c-4902-8efc-b6d41630893a","Type":"ContainerDied","Data":"1a39a3f60bd2b3687ed574178c2c6d5b12ecdc2402d164583207d3598de1c868"}
Jul 01 18:06:59.857998 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.857977    2585 scope.go:117] "RemoveContainer" containerID="90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"
Jul 01 18:06:59.867160 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.867144    2585 scope.go:117] "RemoveContainer" containerID="567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"
Jul 01 18:06:59.874110 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.874092    2585 scope.go:117] "RemoveContainer" containerID="90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"
Jul 01 18:06:59.874338 ip-10-0-139-93 kubenswrapper[2585]: E0701 18:06:59.874321    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01\": container with ID starting with 90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01 not found: ID does not exist" containerID="90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"
Jul 01 18:06:59.874383 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.874346    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01"} err="failed to get container status \"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01\": rpc error: code = NotFound desc = could not find container \"90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01\": container with ID starting with 90cb06ce7eba5b4c7c9892722f6f69ff64e7d95307972a7edce1fd96caa8fe01 not found: ID does not exist"
Jul 01 18:06:59.874383 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.874365    2585 scope.go:117] "RemoveContainer" containerID="567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"
Jul 01 18:06:59.874626 ip-10-0-139-93 kubenswrapper[2585]: E0701 18:06:59.874580    2585 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390\": container with ID starting with 567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390 not found: ID does not exist" containerID="567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"
Jul 01 18:06:59.874724 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.874623    2585 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390"} err="failed to get container status \"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390\": rpc error: code = NotFound desc = could not find container \"567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390\": container with ID starting with 567d755050308ee886ef92534ab04a39fc655e2e84470e9b4e8b46472f904390 not found: ID does not exist"
Jul 01 18:06:59.880468 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.880448    2585 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 18:06:59.884477 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:06:59.884456    2585 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/success-200-isvc-4822a-predictor-59f89c648f-gb6hs"]
Jul 01 18:07:00.546783 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:00.546754    2585 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" path="/var/lib/kubelet/pods/f4d44ed3-a82c-4902-8efc-b6d41630893a/volumes"
Jul 01 18:07:31.912170 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:31.912138    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wqwfq_0496a8d7-d920-4aea-987a-1903d992115b/global-pull-secret-syncer/0.log"
Jul 01 18:07:31.981766 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:31.981739    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8kvfx_fc350dd5-2f49-4d7e-b36b-71bec39ea05a/konnectivity-agent/0.log"
Jul 01 18:07:32.054455 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:32.054429    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-93.ec2.internal_3ca5b1431b790f3e800f6e3e0ef9703f/haproxy/0.log"
Jul 01 18:07:36.416509 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.416469    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/alertmanager/0.log"
Jul 01 18:07:36.446170 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.446148    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/config-reloader/0.log"
Jul 01 18:07:36.469155 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.469128    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/kube-rbac-proxy-web/0.log"
Jul 01 18:07:36.495945 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.495920    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/kube-rbac-proxy/0.log"
Jul 01 18:07:36.518994 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.518971    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/kube-rbac-proxy-metric/0.log"
Jul 01 18:07:36.540670 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.540646    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/prom-label-proxy/0.log"
Jul 01 18:07:36.563572 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.563501    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c73888bd-4b3a-4b08-9c4e-77c452e1cff2/init-config-reloader/0.log"
Jul 01 18:07:36.631947 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.631920    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-state-metrics/0.log"
Jul 01 18:07:36.655781 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.655758    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-rbac-proxy-main/0.log"
Jul 01 18:07:36.676521 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.676479    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-4sxzh_312c257e-97f0-4c9f-b2d6-92f169d44db8/kube-rbac-proxy-self/0.log"
Jul 01 18:07:36.922585 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.922499    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/node-exporter/0.log"
Jul 01 18:07:36.945120 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.945092    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/kube-rbac-proxy/0.log"
Jul 01 18:07:36.972888 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:36.972862    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qcc6m_77d6bee6-8b03-4fbe-b0b8-5abc0b0182c9/init-textfile/0.log"
Jul 01 18:07:37.232693 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:37.232613    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-b8snh_11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d/prometheus-operator/0.log"
Jul 01 18:07:37.261725 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:37.261697    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-b8snh_11abdb0b-7b6c-4582-a903-2d3e2f6d0d6d/kube-rbac-proxy/0.log"
Jul 01 18:07:39.220659 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:39.220574    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/2.log"
Jul 01 18:07:39.224390 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:39.224371    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-q6xtn_1db61671-de63-445c-a5c0-ea12692510d0/console-operator/3.log"
Jul 01 18:07:39.624328 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:39.624285    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-766f4878db-lz56x_7a781b45-8001-448c-a304-a7b9981335e3/console/0.log"
Jul 01 18:07:39.652769 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:39.652749    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-p8l82_7babd4c8-1f91-454e-8ed7-0c6bf7938b40/download-server/0.log"
Jul 01 18:07:40.114515 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.114488    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-rl8t4_f26a7708-bf17-4abb-8b42-d6c3c221b6cd/volume-data-source-validator/0.log"
Jul 01 18:07:40.347584 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347559    2585 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"]
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347891    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347904    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347916    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347922    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347932    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347938    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347944    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347951    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347961    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347966    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kube-rbac-proxy"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347978    2585 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container"
Jul 01 18:07:40.347995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.347983    2585 state_mem.go:107] "Deleted CPUSet assignment" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348031    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kserve-container"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348040    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4d44ed3-a82c-4902-8efc-b6d41630893a" containerName="kube-rbac-proxy"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348046    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kserve-container"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348054    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kserve-container"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348060    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d77ffc0-8d6c-48f1-90ae-abe293506504" containerName="kube-rbac-proxy"
Jul 01 18:07:40.348410 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.348065    2585 memory_manager.go:368] "RemoveStaleState removing state" podUID="17084014-45f0-4920-bfc3-99d6187448ff" containerName="kube-rbac-proxy"
Jul 01 18:07:40.352142 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.352124    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.354823 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.354794    2585 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-49wz8\"/\"default-dockercfg-9fgpq\""
Jul 01 18:07:40.354954 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.354845    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-49wz8\"/\"openshift-service-ca.crt\""
Jul 01 18:07:40.354954 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.354872    2585 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-49wz8\"/\"kube-root-ca.crt\""
Jul 01 18:07:40.360192 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.359692    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"]
Jul 01 18:07:40.403827 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.403761    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz4k8\" (UniqueName: \"kubernetes.io/projected/bd65d50d-b71f-4859-b480-47b169e60d41-kube-api-access-wz4k8\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.403827 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.403801    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-podres\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.403992 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.403890    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-lib-modules\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.403992 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.403922    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-sys\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.403992 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.403949    2585 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-proc\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504516 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504459    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-lib-modules\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504660 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504528    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-sys\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504660 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504549    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-proc\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504660 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504611    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz4k8\" (UniqueName: \"kubernetes.io/projected/bd65d50d-b71f-4859-b480-47b169e60d41-kube-api-access-wz4k8\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504660 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504621    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-lib-modules\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504867 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504666    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-sys\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504867 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504689    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-proc\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504867 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504671    2585 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-podres\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.504867 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.504767    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bd65d50d-b71f-4859-b480-47b169e60d41-podres\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.512551 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.512525    2585 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz4k8\" (UniqueName: \"kubernetes.io/projected/bd65d50d-b71f-4859-b480-47b169e60d41-kube-api-access-wz4k8\") pod \"perf-node-gather-daemonset-k6tsh\" (UID: \"bd65d50d-b71f-4859-b480-47b169e60d41\") " pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.663993 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.663937    2585 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:40.789747 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.789721    2585 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"]
Jul 01 18:07:40.791635 ip-10-0-139-93 kubenswrapper[2585]: W0701 18:07:40.791585    2585 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbd65d50d_b71f_4859_b480_47b169e60d41.slice/crio-3e125b60c3a884fd3ec07fb0c5bf9478fcc4ff12cb5810fc8e17e4a99c185137 WatchSource:0}: Error finding container 3e125b60c3a884fd3ec07fb0c5bf9478fcc4ff12cb5810fc8e17e4a99c185137: Status 404 returned error can't find the container with id 3e125b60c3a884fd3ec07fb0c5bf9478fcc4ff12cb5810fc8e17e4a99c185137
Jul 01 18:07:40.969636 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.969570    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xfvp_d86eb336-78a9-4647-9a6f-e426ca13af76/dns/0.log"
Jul 01 18:07:40.988894 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:40.988873    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7xfvp_d86eb336-78a9-4647-9a6f-e426ca13af76/kube-rbac-proxy/0.log"
Jul 01 18:07:41.047365 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.047343    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh" event={"ID":"bd65d50d-b71f-4859-b480-47b169e60d41","Type":"ContainerStarted","Data":"c2d212c84c90de097f24c96e5f5d670fbc4f83ccc13af4a01ee993853ed08157"}
Jul 01 18:07:41.047460 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.047393    2585 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh" event={"ID":"bd65d50d-b71f-4859-b480-47b169e60d41","Type":"ContainerStarted","Data":"3e125b60c3a884fd3ec07fb0c5bf9478fcc4ff12cb5810fc8e17e4a99c185137"}
Jul 01 18:07:41.047460 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.047422    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:41.062813 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.062776    2585 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh" podStartSLOduration=1.062762938 podStartE2EDuration="1.062762938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 18:07:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 18:07:41.061013847 +0000 UTC m=+3123.013440414" watchObservedRunningTime="2026-07-01 18:07:41.062762938 +0000 UTC m=+3123.015189484"
Jul 01 18:07:41.113171 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.113146    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fgjrz_fe4d7fa6-f0ff-435d-98b8-80e993e6a414/dns-node-resolver/0.log"
Jul 01 18:07:41.578849 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.578817    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86d6c8484-vxf2f_3236e4b5-775e-46b8-ae63-752fb67a26c6/registry/0.log"
Jul 01 18:07:41.625381 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:41.625361    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mw5ct_4832d76a-9f98-422b-adc6-e44703c5248f/node-ca/0.log"
Jul 01 18:07:42.396295 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:42.396264    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6fd9c6d85c-8vt25_addc8041-53ea-41ac-9979-e5872297f067/router/0.log"
Jul 01 18:07:42.800159 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:42.800136    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rvvk6_d8ecef00-bbd7-4c6b-a5bb-53d8c914679c/serve-healthcheck-canary/0.log"
Jul 01 18:07:43.305420 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:43.305395    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknjv_76fa0efc-dc7e-41d3-b077-1ba2afa70be1/kube-rbac-proxy/0.log"
Jul 01 18:07:43.323217 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:43.323197    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknjv_76fa0efc-dc7e-41d3-b077-1ba2afa70be1/exporter/0.log"
Jul 01 18:07:43.340744 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:43.340727    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qknjv_76fa0efc-dc7e-41d3-b077-1ba2afa70be1/extractor/0.log"
Jul 01 18:07:45.358547 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:45.358517    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-848c499849-bcgpt_0c645c55-8652-4a51-9bc1-dc2aeeb27549/manager/0.log"
Jul 01 18:07:45.396108 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:45.396087    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-prz8t_c6a85200-d7b1-42e3-8812-ac80ff1d58d1/server/0.log"
Jul 01 18:07:47.061476 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:47.061448    2585 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-49wz8/perf-node-gather-daemonset-k6tsh"
Jul 01 18:07:49.628051 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:49.628019    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-q6sgz_8f167518-49f7-43b0-909b-c78a72de0220/kube-storage-version-migrator-operator/1.log"
Jul 01 18:07:49.628925 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:49.628906    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-q6sgz_8f167518-49f7-43b0-909b-c78a72de0220/kube-storage-version-migrator-operator/0.log"
Jul 01 18:07:50.605478 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.605450    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/kube-multus-additional-cni-plugins/0.log"
Jul 01 18:07:50.624183 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.624158    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/egress-router-binary-copy/0.log"
Jul 01 18:07:50.641206 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.641177    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/cni-plugins/0.log"
Jul 01 18:07:50.657995 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.657974    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/bond-cni-plugin/0.log"
Jul 01 18:07:50.675173 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.675154    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/routeoverride-cni/0.log"
Jul 01 18:07:50.692710 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.692687    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/whereabouts-cni-bincopy/0.log"
Jul 01 18:07:50.710064 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:50.710045    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7tvs4_b1f7031e-8809-41a2-b19d-3f403e8e1942/whereabouts-cni/0.log"
Jul 01 18:07:51.009931 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:51.009909    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fhj6v_7c07131d-6f49-44bf-acd6-d41b3bb662eb/kube-multus/0.log"
Jul 01 18:07:51.052223 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:51.052199    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2724j_ba2d162e-5b36-4378-a440-669ffe4fc9d4/network-metrics-daemon/0.log"
Jul 01 18:07:51.071196 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:51.071175    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-2724j_ba2d162e-5b36-4378-a440-669ffe4fc9d4/kube-rbac-proxy/0.log"
Jul 01 18:07:52.062190 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.062167    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-controller/0.log"
Jul 01 18:07:52.075266 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.075242    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/1.log"
Jul 01 18:07:52.075431 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.075267    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovn-acl-logging/0.log"
Jul 01 18:07:52.093269 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.093237    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/kube-rbac-proxy-node/0.log"
Jul 01 18:07:52.113416 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.113393    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/kube-rbac-proxy-ovn-metrics/0.log"
Jul 01 18:07:52.126235 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.126219    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/northd/0.log"
Jul 01 18:07:52.144386 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.144365    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/nbdb/0.log"
Jul 01 18:07:52.162733 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.162713    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/sbdb/0.log"
Jul 01 18:07:52.201561 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:52.201533    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qgf8j_af69b820-575a-43d2-89da-70f548839e4f/ovnkube-controller/0.log"
Jul 01 18:07:53.342042 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:53.342014    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-4md7h_b640d14e-3b23-4cd3-9eb1-e8bad24ac69c/check-endpoints/0.log"
Jul 01 18:07:53.389544 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:53.389515    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-kbtzf_6f6d714d-426c-4f54-9c6b-64bd3bc8cd13/network-check-target-container/0.log"
Jul 01 18:07:54.263394 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:54.263334    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-qzsj9_1b3aaaf6-18d1-4929-b45f-2d3b5c6e6564/iptables-alerter/0.log"
Jul 01 18:07:54.848013 ip-10-0-139-93 kubenswrapper[2585]: I0701 18:07:54.847982    2585 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-6vxmz_09d0c844-03fc-4632-ae14-28b3e1097701/tuned/0.log"