Jul 09 15:24:18.043242 ip-10-0-132-248 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 15:24:18.043260 ip-10-0-132-248 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 15:24:18.043270 ip-10-0-132-248 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 15:24:18.043587 ip-10-0-132-248 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 15:24:28.188688 ip-10-0-132-248 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 15:24:28.188707 ip-10-0-132-248 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ddbe309559ed49318784a8bfd7521249 --
Jul 09 15:26:32.195808 ip-10-0-132-248 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 15:26:32.611208 ip-10-0-132-248 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:26:32.611208 ip-10-0-132-248 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 15:26:32.611208 ip-10-0-132-248 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:26:32.611208 ip-10-0-132-248 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 15:26:32.611208 ip-10-0-132-248 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 15:26:32.613042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.612960    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 15:26:32.618765 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618750    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.618765 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618766    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618770    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618773    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618776    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618779    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618782    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618785    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618787    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618790    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618792    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618795    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618797    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618800    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618803    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618805    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618809    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618813    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618817    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618820    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.618824 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618822    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618824    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618827    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618830    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618833    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618836    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618838    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618841    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618843    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618847    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618851    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618853    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618856    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618859    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618861    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618864    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618866    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618868    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618871    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.619292 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618873    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618877    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618879    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618882    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618884    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618886    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618889    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618892    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618894    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618896    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618899    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618901    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618904    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618906    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618909    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618912    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618914    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618917    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618921    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618924    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.619763 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618926    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618929    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618932    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618934    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618936    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618939    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618941    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618944    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618947    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618949    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618952    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618955    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618958    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618961    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618963    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618966    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618968    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618970    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618973    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618975    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.620361 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618978    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618980    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618983    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618985    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618988    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618990    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618993    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618996    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.618999    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619001    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619004    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619006    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619009    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619011    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619014    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619016    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619019    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619021    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619024    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619026    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.620852 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619779    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619787    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619791    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619795    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619798    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619801    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619805    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619807    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619810    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619813    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619816    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619818    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619821    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619825    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619828    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619831    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619833    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619836    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619838    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.621358 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619843    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619846    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619848    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619851    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619853    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619856    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619858    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619861    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619863    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619865    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619868    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619870    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619873    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619875    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619877    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619880    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619882    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619885    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619887    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.621872 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619890    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619893    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619896    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619898    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619901    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619903    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619906    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619908    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619910    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619913    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619915    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619917    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619920    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619923    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619926    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619929    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619932    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619934    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619937    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619939    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.622317 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619942    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619944    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619947    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619949    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619952    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619954    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619956    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619959    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619961    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619964    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619967    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619971    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619973    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619976    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619979    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619981    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619984    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619987    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619989    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.622807 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619993    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619995    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.619998    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620000    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620002    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620005    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620010    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620012    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620015    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620017    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620020    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620022    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620024    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620027    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620030    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620032    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620035    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620037    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620040    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620042    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620045    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.623261 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620047    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620398    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620404    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620408    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620411    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620414    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620416    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620419    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620422    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620424    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620427    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620429    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620431    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620434    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620437    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620439    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620441    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620446    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620448    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620450    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.623789 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620453    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620455    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620458    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620460    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620463    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620465    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620468    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620471    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620473    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620476    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620478    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620481    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620483    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620486    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620488    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620491    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620493    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620496    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620499    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620501    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.624263 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620503    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620506    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620508    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620526    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620528    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620531    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620534    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620536    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620539    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620543    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620546    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620549    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620551    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620554    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620556    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620559    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620561    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620564    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620566    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620569    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.624757 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620571    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620573    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620576    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620579    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620581    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620584    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620586    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620589    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620591    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620594    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620598    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620605    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620608    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620611    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620613    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620616    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620619    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620621    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620624    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.625236 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620627    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620629    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620633    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620635    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620638    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620640    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620643    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620645    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620649    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620652    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620655    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620659    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620661    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620664    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620666    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620669    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620671    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620674    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620677    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.625719 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620679    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.620682    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620757    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620764    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620770    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620776    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620783    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620786    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620790    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620794    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620797    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620800    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620803    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620807    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620809    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620814    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620817    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620819    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620822    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620825    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620829    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620832    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620835    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620839    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 15:26:32.626185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620842    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620845    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620848    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620851    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620854    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620857    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620860    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620863    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620867    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620870    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620873    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620876    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620879    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620883    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620886    2578 flags.go:64] FLAG: --enable-server="true"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620889    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620893    2578 flags.go:64] FLAG: --event-burst="100"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620896    2578 flags.go:64] FLAG: --event-qps="50"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620899    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620902    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620905    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620908    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620911    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620915    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620918    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 09 15:26:32.626754 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620920    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620923    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620926    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620928    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620931    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620934    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620937    2578 flags.go:64] FLAG: --feature-gates=""
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620940    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620943    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620946    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620949    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620951    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620954    2578 flags.go:64] FLAG: --help="false"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620957    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620960    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620963    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620966    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620969    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620972    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620975    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620979    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620981    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620984    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 15:26:32.627372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620988    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620991    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620994    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620997    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.620999    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621002    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621005    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621009    2578 flags.go:64] FLAG: --lock-file=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621011    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621014    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621017    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621022    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621025    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621027    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621030    2578 flags.go:64] FLAG: --logging-format="text"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621033    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621036    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621039    2578 flags.go:64] FLAG: --manifest-url=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621042    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621046    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621049    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621053    2578 flags.go:64] FLAG: --max-pods="110"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621055    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621058    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621061    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 15:26:32.628005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621064    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621067    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621070    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621072    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621080    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621083    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621086    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621089    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621092    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621097    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621100    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621103    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621105    2578 flags.go:64] FLAG: --port="10250"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621108    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621112    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0024892fb3ad8b502"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621115    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621118    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621121    2578 flags.go:64] FLAG: --register-node="true"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621124    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621127    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621130    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621133    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621136    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621139    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621142    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 15:26:32.628619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621145    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621148    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621151    2578 flags.go:64] FLAG: --runonce="false"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621154    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621157    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621160    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621162    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621165    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621168    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621171    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621173    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621176    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621180    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621182    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621185    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621188    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621191    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621194    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621199    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621202    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621205    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621210    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621213    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621215    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621218    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 15:26:32.629224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621221    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621224    2578 flags.go:64] FLAG: --v="2"
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621227    2578 flags.go:64] FLAG: --version="false"
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621231    2578 flags.go:64] FLAG: --vmodule=""
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621234    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.621237    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621345    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621348    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621351    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621354    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621357    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621360    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621363    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621366    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621369    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621372    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621375    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621377    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621379    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621383    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621385    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.629913 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621388    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621390    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621393    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621395    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621398    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621400    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621403    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621406    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621408    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621411    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621413    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621416    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621418    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621420    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621423    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621426    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621429    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621432    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621434    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621436    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.630411 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621439    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621441    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621444    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621446    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621448    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621451    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621454    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621456    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621458    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621461    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621463    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621466    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621468    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621471    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621475    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621479    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621482    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621485    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621488    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.630939 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621491    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621493    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621496    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621499    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621501    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621504    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621506    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621523    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621526    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621528    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621531    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621533    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621536    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621538    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621541    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621544    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621547    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621550    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621553    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.631397 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621556    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621559    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621561    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621564    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621567    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621569    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621572    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621574    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621577    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621580    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621582    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621585    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621588    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621590    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621593    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621595    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621598    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621600    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621603    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621606    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.631877 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621608    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621610    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621613    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621615    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621618    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.621620    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.622251    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.628941    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.628953    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.629723    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.629736    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629798    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629804    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629807    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629810    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.632346 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629812    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629815    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629818    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629820    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629823    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629826    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629828    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629831    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629834    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629837    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629840    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629842    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629845    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629847    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629850    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629852    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629855    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629858    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629860    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629863    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.632771 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629865    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629868    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629870    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629873    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629875    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629878    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629880    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629883    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629885    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629889    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629893    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629896    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629898    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629901    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629904    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629907    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629909    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629912    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629915    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.633247 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629919    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629922    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629926    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629929    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629932    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629935    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629938    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629940    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629942    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629944    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629947    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629949    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629952    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629954    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629956    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629959    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629961    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629963    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629966    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629968    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.633708 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629971    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629973    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629976    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629978    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629980    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629983    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629985    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629987    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629990    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629992    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629995    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.629998    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630000    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630003    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630005    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630008    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630011    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630013    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630016    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630018    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.634246 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630020    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630023    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630026    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630028    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630030    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630033    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630035    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630037    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630040    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630042    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630044    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630047    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630049    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630053    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630057    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.634741 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630059    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.630064    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630170    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630174    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630177    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630180    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630183    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630186    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630188    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630191    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630193    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630196    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630199    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630203    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 15:26:32.635109 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630206    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630208    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630211    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630213    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630216    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630218    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630220    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630223    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630225    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630227    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630230    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630232    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630236    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630239    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630242    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630245    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630249    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630252    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630255    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 15:26:32.635454 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630258    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630260    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630263    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630265    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630268    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630270    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630273    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630275    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630278    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630280    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630282    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630285    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630287    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630290    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630293    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630295    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630297    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630300    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630302    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630305    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 15:26:32.635923 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630307    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630309    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630312    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630314    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630316    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630319    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630321    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630323    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630326    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630328    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630330    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630333    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630336    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630338    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630340    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630343    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630345    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630348    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630350    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630352    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 15:26:32.636418 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630355    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630357    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630359    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630362    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630364    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630367    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630371    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630373    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630375    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630378    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630380    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630383    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630385    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630387    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630390    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630392    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630394    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630396    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630399    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 15:26:32.636909 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630401    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630404    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630406    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630408    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630411    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630414    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630416    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630419    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:32.630421    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.630426    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.630532    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.632616    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.633901    2578 server.go:1019] "Starting client certificate rotation"
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.634003    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 15:26:32.637373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.634063    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 15:26:32.637733 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.636287    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 15:26:32.642089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.642051    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 15:26:32.649973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.649957    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 09 15:26:32.652054 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.652042    2578 log.go:25] "Validated CRI v1 image API"
Jul 09 15:26:32.653723 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.653708    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 15:26:32.657579 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.657560    2578 fs.go:135] Filesystem UUIDs: map[44d61e66-1875-4e36-ba99-fce410b87b93:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8ef5f92b-328e-4f3f-90f0-961a91826698:/dev/nvme0n1p3]
Jul 09 15:26:32.657632 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.657579    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 15:26:32.662587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.662464    2578 manager.go:216] Machine: {Timestamp:2026-07-09 15:26:32.661085346 +0000 UTC m=+0.346814883 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101045 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26d582b42fc97573095a0b90304285 SystemUUID:ec26d582-b42f-c975-7309-5a0b90304285 BootID:ddbe3095-59ed-4931-8784-a8bfd7521249 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9c:a6:8d:a0:a1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9c:a6:8d:a0:a1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:36:d0:a1:4d:ce:f1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 15:26:32.662587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.662586    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 15:26:32.662688 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.662657    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 15:26:32.664483 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.664458    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 15:26:32.664629 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.664487    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-248.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 15:26:32.665227 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.665216    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 15:26:32.665276 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.665229    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 15:26:32.665276 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.665237    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 15:26:32.665865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.665853    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 15:26:32.666663 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.666653    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 15:26:32.667244 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.667229    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 15:26:32.669760 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.669747    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 15:26:32.670045 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670035    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 15:26:32.670089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670067    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 15:26:32.670089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670078    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 15:26:32.670089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670090    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 15:26:32.670203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670098    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 09 15:26:32.670203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670107    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 15:26:32.670470 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670435    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 15:26:32.670504 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.670493    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 15:26:32.672728 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.672711    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 15:26:32.674103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.674090    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 15:26:32.674162 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.674120    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 15:26:32.675237 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675218    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 15:26:32.675342 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675244    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 15:26:32.675459 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675434    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 15:26:32.675506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675462    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 15:26:32.675506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675473    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 15:26:32.675506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675481    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 15:26:32.675506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675490    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 15:26:32.675506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675496    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 15:26:32.675737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675508    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 15:26:32.675737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675533    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 15:26:32.675737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675548    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 15:26:32.675737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.675554    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 15:26:32.676958 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.676943    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 15:26:32.677003 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.676969    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 15:26:32.679731 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.679708    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 15:26:32.679731 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.679725    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-248.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 15:26:32.679861 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.679767    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 15:26:32.680398 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.680384    2578 server.go:1268] "Started kubelet"
Jul 09 15:26:32.680454 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.680434    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 15:26:32.680556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.680490    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 15:26:32.680661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.680570    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 15:26:32.681270 ip-10-0-132-248 systemd[1]: Started Kubernetes Kubelet.
Jul 09 15:26:32.681941 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.681778    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 15:26:32.682410 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.682398    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 15:26:32.684399 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.684382    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 15:26:32.685765 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.685748    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 15:26:32.685836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.685762    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 15:26:32.685898 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.685856    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:32.686597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686578    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 15:26:32.686597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686599    2578 factory.go:55] Registering systemd factory
Jul 09 15:26:32.686776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686607    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 09 15:26:32.686776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686579    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 15:26:32.686776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686648    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 15:26:32.686776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686753    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 15:26:32.686776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686761    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 15:26:32.687057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686791    2578 factory.go:153] Registering CRI-O factory
Jul 09 15:26:32.687057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686804    2578 factory.go:223] Registration of the crio container factory successfully
Jul 09 15:26:32.687057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686835    2578 factory.go:103] Registering Raw factory
Jul 09 15:26:32.687057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.686848    2578 manager.go:1196] Started watching for new ooms in manager
Jul 09 15:26:32.687267 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.687251    2578 manager.go:349] Starting recovery of all containers
Jul 09 15:26:32.687549 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.687538    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 15:26:32.688962 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.688929    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 15:26:32.689630 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.689582    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-248.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 15:26:32.690498 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.688624    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-248.ec2.internal.18c0a890e343df9b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-248.ec2.internal,UID:ip-10-0-132-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-248.ec2.internal,},FirstTimestamp:2026-07-09 15:26:32.680349595 +0000 UTC m=+0.366079132,LastTimestamp:2026-07-09 15:26:32.680349595 +0000 UTC m=+0.366079132,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-248.ec2.internal,}"
Jul 09 15:26:32.692710 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.692606    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rv49l"
Jul 09 15:26:32.697640 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.697620    2578 manager.go:354] Recovery completed
Jul 09 15:26:32.700731 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.700712    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rv49l"
Jul 09 15:26:32.702815 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.702801    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.704627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.704610    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.704728 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.704641    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.704728 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.704655    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.705176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705162    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 15:26:32.705176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705173    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 15:26:32.705272 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705181    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.705272 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705187    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 15:26:32.705852 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705836    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.705903 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705863    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.705903 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.705873    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.706392 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.706380    2578 policy_none.go:49] "None policy: Start"
Jul 09 15:26:32.706430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.706396    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 15:26:32.706430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.706406    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 15:26:32.707182 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.707112    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-248.ec2.internal.18c0a890e4b654c2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-248.ec2.internal,UID:ip-10-0-132-248.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-248.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-248.ec2.internal,},FirstTimestamp:2026-07-09 15:26:32.704627906 +0000 UTC m=+0.390357447,LastTimestamp:2026-07-09 15:26:32.704627906 +0000 UTC m=+0.390357447,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-248.ec2.internal,}"
Jul 09 15:26:32.707590 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.707579    2578 policy_none.go:47] "Start"
Jul 09 15:26:32.747115 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747094    2578 manager.go:337] "Starting Device Plugin manager"
Jul 09 15:26:32.747216 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.747136    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 15:26:32.747216 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747148    2578 server.go:87] "Starting device plugin registration server"
Jul 09 15:26:32.747395 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747381    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 15:26:32.747459 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747397    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 15:26:32.747505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747485    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 15:26:32.747591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747575    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 15:26:32.747591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.747590    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 15:26:32.748123 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.748104    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 15:26:32.748195 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.748148    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:32.847810 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.847784    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.849179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.848788    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.849179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.848815    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.849179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.848824    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.849179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.848851    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.855329 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.855315    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.855389 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.855335    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-248.ec2.internal\": node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:32.871289 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.871243    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:32.872162 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.872143    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 15:26:32.873451 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.873435    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 15:26:32.873522 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.873481    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 15:26:32.873522 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.873503    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 15:26:32.873602 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.873557    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 15:26:32.875560 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.875543    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:26:32.971793 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.971772    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:32.973923 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.973907    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"]
Jul 09 15:26:32.973983 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.973974    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.975410 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.975392    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.975496 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.975420    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.975496 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.975429    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.976779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.976767    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.976902 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.976886    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.976960 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.976917    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.977458 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977441    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.977547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977469    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.977547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977442    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.977547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977482    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.977547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977500    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.977547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.977531    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.978683 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.978665    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.978755 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.978691    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 15:26:32.979433 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.979418    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 15:26:32.979540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.979454    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 15:26:32.979540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:32.979465    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeHasSufficientPID"
Jul 09 15:26:32.994045 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.994025    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-248.ec2.internal\" not found" node="ip-10-0-132-248.ec2.internal"
Jul 09 15:26:32.997363 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:32.997348    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-248.ec2.internal\" not found" node="ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.072540 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.072498    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.089251 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.089225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.089331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.089256    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a6b91f137a83594b4e25554453bfc87-config\") pod \"kube-apiserver-proxy-ip-10-0-132-248.ec2.internal\" (UID: \"9a6b91f137a83594b4e25554453bfc87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.089331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.089272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.173665 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.173605    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.190030 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.189996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.190132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.190043    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.190132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.190073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a6b91f137a83594b4e25554453bfc87-config\") pod \"kube-apiserver-proxy-ip-10-0-132-248.ec2.internal\" (UID: \"9a6b91f137a83594b4e25554453bfc87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.190132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.190098    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.190132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.190117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8a9c7fda06e5bdce7bbaccd6b7c1efb0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal\" (UID: \"8a9c7fda06e5bdce7bbaccd6b7c1efb0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.190132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.190130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a6b91f137a83594b4e25554453bfc87-config\") pod \"kube-apiserver-proxy-ip-10-0-132-248.ec2.internal\" (UID: \"9a6b91f137a83594b4e25554453bfc87\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.274149 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.274126    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.297737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.297707    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.300323 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.300306    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:33.374673 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.374643    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.475226 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.475166    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.575765 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.575745    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.633351 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.633329    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 15:26:33.633914 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.633454    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 15:26:33.676849 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.676825    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.704930 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.704884    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 15:21:33 +0000 UTC" deadline="2028-01-03 21:59:41.787765689 +0000 UTC"
Jul 09 15:26:33.705057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.704931    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13038h33m8.082839065s"
Jul 09 15:26:33.777267 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:33.777204    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:33.793892 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:33.793862    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a9c7fda06e5bdce7bbaccd6b7c1efb0.slice/crio-1918d21fc67cd655f605d0b02b2521d23d694b37da4d9ce96140de540bf6dcb6 WatchSource:0}: Error finding container 1918d21fc67cd655f605d0b02b2521d23d694b37da4d9ce96140de540bf6dcb6: Status 404 returned error can't find the container with id 1918d21fc67cd655f605d0b02b2521d23d694b37da4d9ce96140de540bf6dcb6
Jul 09 15:26:33.794108 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:33.794074    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a6b91f137a83594b4e25554453bfc87.slice/crio-b1732263faa1ab1ffae2ab56918297c2cde5bf4921224e9af9930d52b3b3bbfd WatchSource:0}: Error finding container b1732263faa1ab1ffae2ab56918297c2cde5bf4921224e9af9930d52b3b3bbfd: Status 404 returned error can't find the container with id b1732263faa1ab1ffae2ab56918297c2cde5bf4921224e9af9930d52b3b3bbfd
Jul 09 15:26:33.798197 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.798173    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:26:33.799951 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:33.799934    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:26:35.102891 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:35.102870    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-248.ec2.internal\" not found"
Jul 09 15:26:35.103578 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.103542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal" event={"ID":"8a9c7fda06e5bdce7bbaccd6b7c1efb0","Type":"ContainerStarted","Data":"1918d21fc67cd655f605d0b02b2521d23d694b37da4d9ce96140de540bf6dcb6"}
Jul 09 15:26:35.105096 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.105051    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal" event={"ID":"9a6b91f137a83594b4e25554453bfc87","Type":"ContainerStarted","Data":"b1732263faa1ab1ffae2ab56918297c2cde5bf4921224e9af9930d52b3b3bbfd"}
Jul 09 15:26:35.123481 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.123461    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:26:35.202366 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.202346    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:26:35.211653 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.211631    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:35.230488 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.230469    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 15:26:35.231582 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.231569    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal"
Jul 09 15:26:35.239240 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.239228    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 15:26:35.896582 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.896554    2578 apiserver.go:52] "Watching apiserver"
Jul 09 15:26:35.905167 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.905137    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 15:26:35.906379 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.906352    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-qprmf","openshift-cluster-node-tuning-operator/tuned-296sl","openshift-dns/node-resolver-jj6lk","openshift-image-registry/node-ca-hj47z","openshift-multus/multus-additional-cni-plugins-r9sjn","openshift-network-diagnostics/network-check-target-vg5h6","openshift-network-operator/iptables-alerter-mqvjf","kube-system/global-pull-secret-syncer-h42wx","kube-system/konnectivity-agent-vww99","kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal","openshift-multus/multus-648lm","openshift-multus/network-metrics-daemon-9ktpc"]
Jul 09 15:26:35.908533 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.907965    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:35.909609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.909207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.910478 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.910459    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:35.911428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.911409    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 15:26:35.913316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.912182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:35.913569 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.913551    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.914674 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.914650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:35.914765 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:35.914717    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:35.915203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.915185    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 15:26:35.915458 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.915442    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.916249 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.916232    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:35.918689 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.918633    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.919877 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.918778    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 15:26:35.920003 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.918932    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qhqtv\""
Jul 09 15:26:35.920083 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.920140 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:35.920387 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919117    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.920603 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919158    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5t4zt\""
Jul 09 15:26:35.920700 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919164    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 15:26:35.920775 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919192    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.920851 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919205    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 15:26:35.920900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919222    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.920950 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919266    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9dmpt\""
Jul 09 15:26:35.921001 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919303    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 15:26:35.921049 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919313    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.921049 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919347    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f7rsx\""
Jul 09 15:26:35.921148 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919351    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 15:26:35.921148 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919362    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-x5sng\""
Jul 09 15:26:35.921243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919418    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.921293 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.921246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:35.921293 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919468    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 15:26:35.921471 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919515    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8j8k4\""
Jul 09 15:26:35.921589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.919792    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 15:26:35.922033 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.922015    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:35.922114 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:35.922078    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:35.924228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.924206    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ts9cm\""
Jul 09 15:26:35.925470 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-conf-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.925567 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925487    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c762516-924f-47e5-9db9-2d92680a5c6a-serviceca\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:35.925567 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925511    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.925567 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-os-release\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.925726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925596    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.925726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925631    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.925726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-host-slash\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:35.925726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925709    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-multus\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925725    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925792    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnnnq\" (UniqueName: \"kubernetes.io/projected/66df486a-05c7-4e72-b2e0-bce404d9ccac-kube-api-access-mnnnq\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c762516-924f-47e5-9db9-2d92680a5c6a-host\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-socket-dir-parent\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.925949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925899    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-system-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.925978    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-cnibin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-os-release\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-bin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk8z7\" (UniqueName: \"kubernetes.io/projected/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-kube-api-access-rk8z7\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-hostroot\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-multus-daemon-config\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.926221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926186    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 15:26:35.926544 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926290    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:35.926599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926572    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 15:26:35.927203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.926190    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-multus-certs\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927294 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-etc-kubernetes\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927294 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cnibin\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.927394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.927394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4sm9\" (UniqueName: \"kubernetes.io/projected/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-kube-api-access-g4sm9\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:35.927394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-cni-binary-copy\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927427    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-netns\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-kubelet\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927486    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-k8s-cni-cncf-io\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-iptables-alerter-script\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:35.927744 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927538    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66df486a-05c7-4e72-b2e0-bce404d9ccac-hosts-file\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:35.927744 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtqrc\" (UniqueName: \"kubernetes.io/projected/950abca6-2db7-4906-9089-dece19bf4781-kube-api-access-rtqrc\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:35.927744 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66df486a-05c7-4e72-b2e0-bce404d9ccac-tmp-dir\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:35.927744 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9hk8\" (UniqueName: \"kubernetes.io/projected/6c762516-924f-47e5-9db9-2d92680a5c6a-kube-api-access-f9hk8\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:35.928001 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.927983    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.928049 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.928023    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-glwjs\""
Jul 09 15:26:35.928301 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.928287    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 15:26:35.928379 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.928363    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.929302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.929284    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 15:26:35.929824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.929669    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.929824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.929673    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 15:26:35.929824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.929812    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 15:26:35.930025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.930005    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-6qwrl\""
Jul 09 15:26:35.930136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.930123    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 15:26:35.932829 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.932813    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:35.933136 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:35.933111    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:35.936734 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.936710    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 15:26:35.949190 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.949170    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gx2h7"
Jul 09 15:26:35.957012 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:35.956994    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gx2h7"
Jul 09 15:26:36.014178 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.014153    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 15:26:36.027972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.027946    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-netns\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.028089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.027988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-device-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.028089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028016    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.028089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028042    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-k8s-cni-cncf-io\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.028089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-iptables-alerter-script\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028091    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-registration-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-netd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028139    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-tmp\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bbwj\" (UniqueName: \"kubernetes.io/projected/629de1d7-d62d-4902-a346-4747115489b5-kube-api-access-8bbwj\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtqrc\" (UniqueName: \"kubernetes.io/projected/950abca6-2db7-4906-9089-dece19bf4781-kube-api-access-rtqrc\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-sys-fs\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.028252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028267    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-systemd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-conf-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-ovn\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-script-lib\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-modprobe-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028372    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysconfig\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-multus\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028425    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-socket-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-systemd-units\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-run\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-sys\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.028505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnnnq\" (UniqueName: \"kubernetes.io/projected/66df486a-05c7-4e72-b2e0-bce404d9ccac-kube-api-access-mnnnq\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c762516-924f-47e5-9db9-2d92680a5c6a-host\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028542    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-system-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028066    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-netns\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028626    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-k8s-cni-cncf-io\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-cnibin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-iptables-alerter-script\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6c762516-924f-47e5-9db9-2d92680a5c6a-host\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-system-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-cnibin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-multus\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028907    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-conf-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-hostroot\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-multus-certs\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-hostroot\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.028995    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cnibin\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-dbus\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.029245 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk8z7\" (UniqueName: \"kubernetes.io/projected/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-kube-api-access-rk8z7\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-run-multus-certs\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-kubelet-config\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029127    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg86s\" (UniqueName: \"kubernetes.io/projected/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kube-api-access-zg86s\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029199    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-konnectivity-ca\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029210    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-binary-copy\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029219    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-node-log\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-config\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-var-lib-kubelet\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029314    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cnibin\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029387    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5803a7-2750-43cb-89bd-4fea94ea4898-ovn-node-metrics-cert\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlxs6\" (UniqueName: \"kubernetes.io/projected/3d446ecc-426d-465e-96b6-d42360901d7f-kube-api-access-vlxs6\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029417    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-cni-binary-copy\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029437    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-kubelet\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.029942 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029453    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-cni-dir\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-kubelet\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-netns\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029577    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-var-lib-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029602    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66df486a-05c7-4e72-b2e0-bce404d9ccac-hosts-file\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029624    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-host\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029643    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66df486a-05c7-4e72-b2e0-bce404d9ccac-tmp-dir\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029671    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9hk8\" (UniqueName: \"kubernetes.io/projected/6c762516-924f-47e5-9db9-2d92680a5c6a-kube-api-access-f9hk8\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-conf\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66df486a-05c7-4e72-b2e0-bce404d9ccac-hosts-file\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029715    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-agent-certs\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c762516-924f-47e5-9db9-2d92680a5c6a-serviceca\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-os-release\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029878    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.030814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.029939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-cni-binary-copy\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-host-slash\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-os-release\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-slash\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030048    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66df486a-05c7-4e72-b2e0-bce404d9ccac-tmp-dir\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030055    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-system-cni-dir\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-host-slash\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-log-socket\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-env-overrides\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030578    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-kubernetes\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-systemd\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030658    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-etc-tuned\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-socket-dir-parent\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-os-release\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-bin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-multus-socket-dir-parent\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6c762516-924f-47e5-9db9-2d92680a5c6a-serviceca\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.031529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030826    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-os-release\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-host-var-lib-cni-bin\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030843    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-etc-kubernetes\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030922    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/950abca6-2db7-4906-9089-dece19bf4781-etc-kubernetes\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-kubelet\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.030981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-multus-daemon-config\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4sm9\" (UniqueName: \"kubernetes.io/projected/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-kube-api-access-g4sm9\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031036    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-bin\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmd85\" (UniqueName: \"kubernetes.io/projected/de5803a7-2750-43cb-89bd-4fea94ea4898-kube-api-access-lmd85\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-lib-modules\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031191    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-etc-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/950abca6-2db7-4906-9089-dece19bf4781-multus-daemon-config\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.032164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.031433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.036385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.036364    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 15:26:36.040008 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.039956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnnnq\" (UniqueName: \"kubernetes.io/projected/66df486a-05c7-4e72-b2e0-bce404d9ccac-kube-api-access-mnnnq\") pod \"node-resolver-jj6lk\" (UID: \"66df486a-05c7-4e72-b2e0-bce404d9ccac\") " pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.040087 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.040010    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk8z7\" (UniqueName: \"kubernetes.io/projected/a16510d5-86e7-40f6-aa55-b5ed60ec4fe7-kube-api-access-rk8z7\") pod \"iptables-alerter-mqvjf\" (UID: \"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7\") " pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.040087 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.040037    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtqrc\" (UniqueName: \"kubernetes.io/projected/950abca6-2db7-4906-9089-dece19bf4781-kube-api-access-rtqrc\") pod \"multus-648lm\" (UID: \"950abca6-2db7-4906-9089-dece19bf4781\") " pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.040697 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.040671    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9hk8\" (UniqueName: \"kubernetes.io/projected/6c762516-924f-47e5-9db9-2d92680a5c6a-kube-api-access-f9hk8\") pod \"node-ca-hj47z\" (UID: \"6c762516-924f-47e5-9db9-2d92680a5c6a\") " pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.041397 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.041372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4sm9\" (UniqueName: \"kubernetes.io/projected/66e56408-1d75-4c3b-ac75-bf0c62ee4f07-kube-api-access-g4sm9\") pod \"multus-additional-cni-plugins-r9sjn\" (UID: \"66e56408-1d75-4c3b-ac75-bf0c62ee4f07\") " pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.132003 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.131969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-slash\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132010    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-log-socket\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132055    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-env-overrides\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132067    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-log-socket\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-kubernetes\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-slash\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132114    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-systemd\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-etc-tuned\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132161    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132181    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-kubelet\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-kubernetes\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132205    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132198    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-systemd\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-bin\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmd85\" (UniqueName: \"kubernetes.io/projected/de5803a7-2750-43cb-89bd-4fea94ea4898-kube-api-access-lmd85\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132276    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-kubelet\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132298    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-lib-modules\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-etc-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-bin\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-etc-selinux\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132344    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-device-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132380    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132416    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132430    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-device-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.132565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132463    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-registration-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132489    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-netd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-lib-modules\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-tmp\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132545    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-etc-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bbwj\" (UniqueName: \"kubernetes.io/projected/629de1d7-d62d-4902-a346-4747115489b5-kube-api-access-8bbwj\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132595    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-sys-fs\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.132600    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-registration-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132657    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-systemd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.132695    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:26:36.632665612 +0000 UTC m=+3.093156606 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-systemd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-ovn\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132769    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-script-lib\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-modprobe-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.133382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysconfig\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-ovn\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-sys-fs\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132882    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-cni-netd\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132940    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-modprobe-d\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132597    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-env-overrides\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132845    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-socket-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysconfig\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.132992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-socket-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-systemd-units\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133038    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-run\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133059    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-sys\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-dbus\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-kubelet-config\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133120    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-run\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133138    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg86s\" (UniqueName: \"kubernetes.io/projected/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kube-api-access-zg86s\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.134180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133162    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-konnectivity-ca\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-node-log\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133213    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-sys\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-config\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133252    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-var-lib-kubelet\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5803a7-2750-43cb-89bd-4fea94ea4898-ovn-node-metrics-cert\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133323    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlxs6\" (UniqueName: \"kubernetes.io/projected/3d446ecc-426d-465e-96b6-d42360901d7f-kube-api-access-vlxs6\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-dbus\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133348    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133391    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-netns\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.133400    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-var-lib-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.133450    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:36.633433788 +0000 UTC m=+3.093924764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-host\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-host-run-netns\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.134849 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133513    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-conf\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-host\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133447    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-script-lib\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-agent-certs\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133163    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-systemd-units\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133584    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-var-lib-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-run-openvswitch\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-etc-sysctl-conf\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/629de1d7-d62d-4902-a346-4747115489b5-var-lib-kubelet\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/de5803a7-2750-43cb-89bd-4fea94ea4898-node-log\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133251    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6660f188-abdc-44da-8487-cadab2eb96e7-kubelet-config\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.133958    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-konnectivity-ca\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.135599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.134209    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/de5803a7-2750-43cb-89bd-4fea94ea4898-ovnkube-config\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.136140 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.135841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-tmp\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.136382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.136361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/04ca77ba-3e4d-4bb5-8522-ee755d697ab9-agent-certs\") pod \"konnectivity-agent-vww99\" (UID: \"04ca77ba-3e4d-4bb5-8522-ee755d697ab9\") " pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.136671 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.136650    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/de5803a7-2750-43cb-89bd-4fea94ea4898-ovn-node-metrics-cert\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.137123 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.137108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/629de1d7-d62d-4902-a346-4747115489b5-etc-tuned\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.141800 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.141778    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmd85\" (UniqueName: \"kubernetes.io/projected/de5803a7-2750-43cb-89bd-4fea94ea4898-kube-api-access-lmd85\") pod \"ovnkube-node-qprmf\" (UID: \"de5803a7-2750-43cb-89bd-4fea94ea4898\") " pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.142524 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.142500    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:36.142613 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.142539    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:36.142613 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.142562    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:36.142721 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.142628    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:36.642598874 +0000 UTC m=+3.103089852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:36.142991 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.142970    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bbwj\" (UniqueName: \"kubernetes.io/projected/629de1d7-d62d-4902-a346-4747115489b5-kube-api-access-8bbwj\") pod \"tuned-296sl\" (UID: \"629de1d7-d62d-4902-a346-4747115489b5\") " pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.145287 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.145265    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlxs6\" (UniqueName: \"kubernetes.io/projected/3d446ecc-426d-465e-96b6-d42360901d7f-kube-api-access-vlxs6\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.145535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.145515    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg86s\" (UniqueName: \"kubernetes.io/projected/4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b-kube-api-access-zg86s\") pod \"aws-ebs-csi-driver-node-c28np\" (UID: \"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.229638 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.229568    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mqvjf"
Jul 09 15:26:36.229638 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.229600    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 15:26:36.239538 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.239518    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-648lm"
Jul 09 15:26:36.246169 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.246145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jj6lk"
Jul 09 15:26:36.252883 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.252866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hj47z"
Jul 09 15:26:36.261394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.261373    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r9sjn"
Jul 09 15:26:36.268018 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.267992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:36.275820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.275802    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:26:36.284463 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.284446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-296sl"
Jul 09 15:26:36.291049 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.291028    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np"
Jul 09 15:26:36.636764 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.636722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:36.636935 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.636791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:36.636935 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.636889    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:36.636935 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.636913    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:36.637092 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.636963    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:37.636941848 +0000 UTC m=+4.097432832 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:36.637092 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.636987    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:26:37.636975973 +0000 UTC m=+4.097466952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:36.737124 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.737094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:36.737276 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.737252    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:36.737346 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.737277    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:36.737346 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.737290    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:36.737346 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:36.737344    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:37.737330935 +0000 UTC m=+4.197821911 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:36.959306 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.959233    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 15:21:35 +0000 UTC" deadline="2028-04-06 16:31:06.35448997 +0000 UTC"
Jul 09 15:26:36.959306 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:36.959266    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15289h4m29.395230423s"
Jul 09 15:26:37.009888 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.009856    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ca77ba_3e4d_4bb5_8522_ee755d697ab9.slice/crio-0cae79bfc724ad867ffbf879e17c2ef0e39748d97acfef1f0c78bd8a00342843 WatchSource:0}: Error finding container 0cae79bfc724ad867ffbf879e17c2ef0e39748d97acfef1f0c78bd8a00342843: Status 404 returned error can't find the container with id 0cae79bfc724ad867ffbf879e17c2ef0e39748d97acfef1f0c78bd8a00342843
Jul 09 15:26:37.010895 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.010865    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c762516_924f_47e5_9db9_2d92680a5c6a.slice/crio-696ae9ad2033da97cd563210e4265f7cf25147e2db2293a9dd59d5a444e965cb WatchSource:0}: Error finding container 696ae9ad2033da97cd563210e4265f7cf25147e2db2293a9dd59d5a444e965cb: Status 404 returned error can't find the container with id 696ae9ad2033da97cd563210e4265f7cf25147e2db2293a9dd59d5a444e965cb
Jul 09 15:26:37.011496 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.011458    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66df486a_05c7_4e72_b2e0_bce404d9ccac.slice/crio-7930c944c823709c22eb24fd0e7433c40d91a9f8f1832e8225a14cc5702324ab WatchSource:0}: Error finding container 7930c944c823709c22eb24fd0e7433c40d91a9f8f1832e8225a14cc5702324ab: Status 404 returned error can't find the container with id 7930c944c823709c22eb24fd0e7433c40d91a9f8f1832e8225a14cc5702324ab
Jul 09 15:26:37.012712 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.012554    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950abca6_2db7_4906_9089_dece19bf4781.slice/crio-0e5644100ccf23c72e0d89b83c4f849f7fe716436091c98e5460b168977b3378 WatchSource:0}: Error finding container 0e5644100ccf23c72e0d89b83c4f849f7fe716436091c98e5460b168977b3378: Status 404 returned error can't find the container with id 0e5644100ccf23c72e0d89b83c4f849f7fe716436091c98e5460b168977b3378
Jul 09 15:26:37.017166 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.016349    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb85410_bcc1_472d_a2c0_71b5d1ecaf0b.slice/crio-ef979fec73a38effed929961544c09264fe3a3e180887a7e474e91b96f9353fe WatchSource:0}: Error finding container ef979fec73a38effed929961544c09264fe3a3e180887a7e474e91b96f9353fe: Status 404 returned error can't find the container with id ef979fec73a38effed929961544c09264fe3a3e180887a7e474e91b96f9353fe
Jul 09 15:26:37.021545 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.021524    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde5803a7_2750_43cb_89bd_4fea94ea4898.slice/crio-ff83d546d6ffa1037984b2a05771fca725fa2b690453e8a193ba54bdddc05ba1 WatchSource:0}: Error finding container ff83d546d6ffa1037984b2a05771fca725fa2b690453e8a193ba54bdddc05ba1: Status 404 returned error can't find the container with id ff83d546d6ffa1037984b2a05771fca725fa2b690453e8a193ba54bdddc05ba1
Jul 09 15:26:37.023442 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:26:37.023402    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda16510d5_86e7_40f6_aa55_b5ed60ec4fe7.slice/crio-4de4f99d1121a4bad4828c3c08fc7b7d9b9dc3cf542c5f59f04583fbb9f4c116 WatchSource:0}: Error finding container 4de4f99d1121a4bad4828c3c08fc7b7d9b9dc3cf542c5f59f04583fbb9f4c116: Status 404 returned error can't find the container with id 4de4f99d1121a4bad4828c3c08fc7b7d9b9dc3cf542c5f59f04583fbb9f4c116
Jul 09 15:26:37.099869 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.099699    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:37.099986 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.099964    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:37.115794 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.115771    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vww99" event={"ID":"04ca77ba-3e4d-4bb5-8522-ee755d697ab9","Type":"ContainerStarted","Data":"0cae79bfc724ad867ffbf879e17c2ef0e39748d97acfef1f0c78bd8a00342843"}
Jul 09 15:26:37.117697 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.117676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"ff83d546d6ffa1037984b2a05771fca725fa2b690453e8a193ba54bdddc05ba1"}
Jul 09 15:26:37.119237 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.119208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerStarted","Data":"22e786ae216169a80b4a5f4b623ad8429b0f7e0da874a189baebda651d41f865"}
Jul 09 15:26:37.120223 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.120199    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-296sl" event={"ID":"629de1d7-d62d-4902-a346-4747115489b5","Type":"ContainerStarted","Data":"725d5359d7b95a97b00ddd981da48f841b367e4e9b599d143638d8315291e0cd"}
Jul 09 15:26:37.123591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.122679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-648lm" event={"ID":"950abca6-2db7-4906-9089-dece19bf4781","Type":"ContainerStarted","Data":"0e5644100ccf23c72e0d89b83c4f849f7fe716436091c98e5460b168977b3378"}
Jul 09 15:26:37.125545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.125521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jj6lk" event={"ID":"66df486a-05c7-4e72-b2e0-bce404d9ccac","Type":"ContainerStarted","Data":"7930c944c823709c22eb24fd0e7433c40d91a9f8f1832e8225a14cc5702324ab"}
Jul 09 15:26:37.128119 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.128097    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hj47z" event={"ID":"6c762516-924f-47e5-9db9-2d92680a5c6a","Type":"ContainerStarted","Data":"696ae9ad2033da97cd563210e4265f7cf25147e2db2293a9dd59d5a444e965cb"}
Jul 09 15:26:37.130148 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.130130    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal" event={"ID":"9a6b91f137a83594b4e25554453bfc87","Type":"ContainerStarted","Data":"5f861b5f0d6b03d70af1219c26f77d3a667be2e1332f277b7d48c3494ae753ae"}
Jul 09 15:26:37.136024 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.136002    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mqvjf" event={"ID":"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7","Type":"ContainerStarted","Data":"4de4f99d1121a4bad4828c3c08fc7b7d9b9dc3cf542c5f59f04583fbb9f4c116"}
Jul 09 15:26:37.137162 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.137138    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np" event={"ID":"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b","Type":"ContainerStarted","Data":"ef979fec73a38effed929961544c09264fe3a3e180887a7e474e91b96f9353fe"}
Jul 09 15:26:37.148220 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.148183    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-248.ec2.internal" podStartSLOduration=2.148171192 podStartE2EDuration="2.148171192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:37.147878645 +0000 UTC m=+3.608369716" watchObservedRunningTime="2026-07-09 15:26:37.148171192 +0000 UTC m=+3.608662189"
Jul 09 15:26:37.642846 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.642815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:37.643013 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.642905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:37.643118 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.643015    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:37.643118 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.643067    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:39.643050038 +0000 UTC m=+6.103541022 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:37.643446 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.643427    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:37.643528 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.643481    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:26:39.643466144 +0000 UTC m=+6.103957122 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:37.744128 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.744095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:37.744339 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.744306    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:37.744339 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.744328    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:37.744482 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.744342    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:37.744482 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:37.744405    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:39.744384143 +0000 UTC m=+6.204875138 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:37.960221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.960117    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 15:21:35 +0000 UTC" deadline="2028-02-05 19:53:03.849130088 +0000 UTC"
Jul 09 15:26:37.960221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:37.960177    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13828h26m25.888957778s"
Jul 09 15:26:38.101557 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:38.101520    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:38.101720 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:38.101665    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:38.102099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:38.102078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:38.102209 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:38.102187    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:38.148009 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:38.146565    2578 generic.go:356] "Generic (PLEG): container finished" podID="8a9c7fda06e5bdce7bbaccd6b7c1efb0" containerID="6e7e0d26a9946bdeaeb3367b398a9bf4312b980da9d9a015c486d2b5106f89d5" exitCode=0
Jul 09 15:26:38.148009 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:38.147734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal" event={"ID":"8a9c7fda06e5bdce7bbaccd6b7c1efb0","Type":"ContainerDied","Data":"6e7e0d26a9946bdeaeb3367b398a9bf4312b980da9d9a015c486d2b5106f89d5"}
Jul 09 15:26:39.099123 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.099089    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:39.099859 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.099219    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:39.156003 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.155968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal" event={"ID":"8a9c7fda06e5bdce7bbaccd6b7c1efb0","Type":"ContainerStarted","Data":"baee8cde37f8df0aab3934542c90e01a8c95fa2f7ac4d1b9fc6c906a34713037"}
Jul 09 15:26:39.174228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.174158    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-248.ec2.internal" podStartSLOduration=4.174138516 podStartE2EDuration="4.174138516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:39.173884614 +0000 UTC m=+5.634375613" watchObservedRunningTime="2026-07-09 15:26:39.174138516 +0000 UTC m=+5.634629514"
Jul 09 15:26:39.664920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.664882    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:39.665104 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.664966    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:39.665104 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.665084    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:39.665228 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.665139    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:43.665122423 +0000 UTC m=+10.125613401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:39.665531 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.665510    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:39.665597 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.665569    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:26:43.665552969 +0000 UTC m=+10.126043945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:39.765835 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:39.765801    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:39.766017 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.765970    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:39.766017 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.765989    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:39.766017 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.766002    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:39.766186 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:39.766064    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:43.766044855 +0000 UTC m=+10.226535853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:40.099907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:40.099873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:40.100335 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:40.100024    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:40.110268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:40.110238    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:40.110675 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:40.110648    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:41.099353 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:41.099319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:41.099563 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:41.099431    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:42.101648 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:42.101181    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:42.101648 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:42.101310    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:42.102167 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:42.102144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:42.102260 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:42.102234    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:43.099069 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:43.099040    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:43.099231 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.099147    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:43.704442 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:43.704408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:43.704931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:43.704466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:43.704931 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.704585    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:43.704931 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.704639    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:51.704624802 +0000 UTC m=+18.165115780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:43.705103 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.705010    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:43.705103 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.705056    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:26:51.705041651 +0000 UTC m=+18.165532640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:43.805393 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:43.805235    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:43.805554 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.805406    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:43.805554 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.805429    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:43.805554 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.805448    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:43.805554 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:43.805513    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:26:51.805492191 +0000 UTC m=+18.265983169 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:44.100722 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:44.100087    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:44.100722 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:44.100210    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:44.100722 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:44.100574    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:44.100722 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:44.100675    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:45.099569 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:45.099535    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:45.100105 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:45.099671    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:46.100975 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:46.100895    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:46.101424 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:46.101033    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:46.101424 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:46.101103    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:46.101424 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:46.101219    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:47.099882 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:47.099852    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:47.100068 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:47.099967    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:48.099965 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:48.099928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:48.100408 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:48.099939    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:48.100408 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:48.100072    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:48.100408 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:48.100137    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:49.099331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:49.099294    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:49.099518 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:49.099410    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:50.099682 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:50.099647    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:50.100154 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:50.099809    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:50.100154 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:50.099872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:50.100154 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:50.099978    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:51.099385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:51.099348    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:51.099589 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.099466    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:51.765685 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:51.765639    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:51.766114 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:51.765710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:51.766114 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.765785    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:51.766114 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.765852    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:51.766114 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.765857    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:07.765836196 +0000 UTC m=+34.226327181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:26:51.766114 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.765919    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:27:07.765901525 +0000 UTC m=+34.226392514 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:26:51.866576 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:51.866486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:51.866732 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.866634    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:26:51.866732 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.866657    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:26:51.866732 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.866670    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:51.866963 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:51.866732    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:07.866712226 +0000 UTC m=+34.327203260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:26:52.099198 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:52.099118    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:52.099366 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:52.099265    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:52.099366 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:52.099341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:52.099469 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:52.099450    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:53.099177 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:53.099145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:53.099651 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:53.099260    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:54.100109 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:54.100071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:54.100552 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:54.100200    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:54.100552 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:54.100253    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:54.100552 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:54.100358    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:55.099369 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:55.099329    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:55.099551 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:55.099441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:56.098971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:56.098935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:56.099331 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:56.099062    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:56.099331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:56.099119    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:56.099331 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:56.099207    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:57.100152 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.099820    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:57.100765 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:57.100189    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:57.192143 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.192106    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="1c16613beb43611692dbd46acc6a7446478e148cccb85e2714207a2eb8c4966c" exitCode=0
Jul 09 15:26:57.192320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.192178    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"1c16613beb43611692dbd46acc6a7446478e148cccb85e2714207a2eb8c4966c"}
Jul 09 15:26:57.194145 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.194116    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-296sl" event={"ID":"629de1d7-d62d-4902-a346-4747115489b5","Type":"ContainerStarted","Data":"09f4d061125052cc916e3572cfb7c8d263a9017ddf0098fbaac5153be5191160"}
Jul 09 15:26:57.195826 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.195800    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-648lm" event={"ID":"950abca6-2db7-4906-9089-dece19bf4781","Type":"ContainerStarted","Data":"1302e1c648abdf24a65704e03671ac62c6b1e15bb58a83f0b87ff10b9ab22fcb"}
Jul 09 15:26:57.197399 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.197370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jj6lk" event={"ID":"66df486a-05c7-4e72-b2e0-bce404d9ccac","Type":"ContainerStarted","Data":"b15b24e3cb5d6b8506351b0e5db1fad913a7131b92cf3aa8db6a1c2923571b4d"}
Jul 09 15:26:57.199106 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.199086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hj47z" event={"ID":"6c762516-924f-47e5-9db9-2d92680a5c6a","Type":"ContainerStarted","Data":"c22284bd0e21b41c0b05b823015656bb24e2acf0b07a9874626cc3f9e7a2f12e"}
Jul 09 15:26:57.200648 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.200623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np" event={"ID":"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b","Type":"ContainerStarted","Data":"a81f153e54927d07ff5a3ed8cf6facc83c3291be54d47e29cb99bd82336ff3e3"}
Jul 09 15:26:57.202257 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.202231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vww99" event={"ID":"04ca77ba-3e4d-4bb5-8522-ee755d697ab9","Type":"ContainerStarted","Data":"a03fcca48a1fee0dd9720bea81c677ac492a01a3ce6f290f6f37eef0a74ba3fe"}
Jul 09 15:26:57.205034 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205016    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:26:57.205368 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205344    2578 generic.go:356] "Generic (PLEG): container finished" podID="de5803a7-2750-43cb-89bd-4fea94ea4898" containerID="29783c0a2d6a33cfedf6f79690685e5581ceb4487b0141dfa8a80464fc3e8df1" exitCode=1
Jul 09 15:26:57.205449 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"fe7c790838a7e229a41a12fc022ad38beffae547f28813efad2fbb989950cf43"}
Jul 09 15:26:57.205497 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"6b3b016497432747a4a15265a8e038f02183d7ebc7e549f2e312bd5d90ca88c5"}
Jul 09 15:26:57.205551 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205499    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"74e6844db7cb8e1e1b6467925ebeaa43917687909aa143eb5a00f0df886d313a"}
Jul 09 15:26:57.205602 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"5b7232e80415bd52682d0041404b089fffe5fba795af98c2d7cc23b7991c5c0b"}
Jul 09 15:26:57.205602 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205592    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerDied","Data":"29783c0a2d6a33cfedf6f79690685e5581ceb4487b0141dfa8a80464fc3e8df1"}
Jul 09 15:26:57.205671 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.205624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"d7f171f270a73948da8a837a230c8ea409db7e96ca33d5598f65a7691049316f"}
Jul 09 15:26:57.276518 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.276447    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-648lm" podStartSLOduration=3.9808048879999998 podStartE2EDuration="23.276431943s" totalImagesPullingTime="19.295627055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:57.255985769 +0000 UTC m=+23.716476769" watchObservedRunningTime="2026-07-09 15:26:57.276431943 +0000 UTC m=+23.736922939"
Jul 09 15:26:57.276819 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.276793    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jj6lk" podStartSLOduration=4.105293855 podStartE2EDuration="23.276781954s" totalImagesPullingTime="19.171488099s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:57.276159501 +0000 UTC m=+23.736650500" watchObservedRunningTime="2026-07-09 15:26:57.276781954 +0000 UTC m=+23.737272954"
Jul 09 15:26:57.297815 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.297771    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hj47z" podStartSLOduration=4.126060787 podStartE2EDuration="23.297743409s" totalImagesPullingTime="19.171682622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:57.297386521 +0000 UTC m=+23.757877520" watchObservedRunningTime="2026-07-09 15:26:57.297743409 +0000 UTC m=+23.758234406"
Jul 09 15:26:57.314480 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.314426    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vww99" podStartSLOduration=12.583490765 podStartE2EDuration="23.314414178s" totalImagesPullingTime="10.730923413s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:57.31413608 +0000 UTC m=+23.774627079" watchObservedRunningTime="2026-07-09 15:26:57.314414178 +0000 UTC m=+23.774905175"
Jul 09 15:26:57.338932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.338890    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-296sl" podStartSLOduration=4.153573504 podStartE2EDuration="23.338878575s" totalImagesPullingTime="19.185305071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:57.338124154 +0000 UTC m=+23.798615150" watchObservedRunningTime="2026-07-09 15:26:57.338878575 +0000 UTC m=+23.799369571"
Jul 09 15:26:57.961268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.961243    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 15:26:57.985898 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.985810    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T15:26:57.961265282Z","UUID":"b742bb48-4210-463c-bbf7-e857243df1b6","Handler":null,"Name":"","Endpoint":""}
Jul 09 15:26:57.987481 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.987464    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 15:26:57.987613 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:57.987502    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 15:26:58.099949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:58.099864    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:26:58.100119 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:58.099870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:26:58.100119 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:58.100003    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:26:58.100119 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:58.100074    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:26:58.209375 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:58.209338    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mqvjf" event={"ID":"a16510d5-86e7-40f6-aa55-b5ed60ec4fe7","Type":"ContainerStarted","Data":"3cb788403b565a0ba0edfd576e2a5ad2144e2298ea42389bd5336c9a1b155adf"}
Jul 09 15:26:58.211551 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:58.211505    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np" event={"ID":"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b","Type":"ContainerStarted","Data":"558aaa1da9385ba7cd91aabb7e6fad926428b1c4fa3d2488ced5f650c27317b5"}
Jul 09 15:26:58.231347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:58.231302    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mqvjf" podStartSLOduration=5.053408914 podStartE2EDuration="24.231286935s" totalImagesPullingTime="19.177878021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:26:58.230983563 +0000 UTC m=+24.691474561" watchObservedRunningTime="2026-07-09 15:26:58.231286935 +0000 UTC m=+24.691777961"
Jul 09 15:26:59.099623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:59.099579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:26:59.099806 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:26:59.099687    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:26:59.217889 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:59.217716    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:26:59.218934 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:59.218465    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"af66692fd05689db70523c38e7a133b8296f1c9569b59dad44a1cebbd6621991"}
Jul 09 15:26:59.407362 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:59.407330    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:26:59.407943 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:26:59.407905    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:27:00.099972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.099935    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:00.099972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.099970    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:00.100249 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:00.100064    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:00.100249 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:00.100177    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:00.222986 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.222936    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np" event={"ID":"4cb85410-bcc1-472d-a2c0-71b5d1ecaf0b","Type":"ContainerStarted","Data":"4eef1790570ec1bf7b6a806549668243e63e8c0c83674348b074b611a47cdc77"}
Jul 09 15:27:00.223526 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.223190    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:27:00.223659 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.223646    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vww99"
Jul 09 15:27:00.246283 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:00.246239    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c28np" podStartSLOduration=4.179978843 podStartE2EDuration="26.246225985s" totalImagesPullingTime="22.066247142s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:00.2458979 +0000 UTC m=+26.706388900" watchObservedRunningTime="2026-07-09 15:27:00.246225985 +0000 UTC m=+26.706716981"
Jul 09 15:27:01.099148 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:01.099114    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:01.099325 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:01.099237    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:02.099961 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.099678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:02.100526 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.099793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:02.100526 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:02.100087    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:02.100526 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:02.100109    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:02.230090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.230068    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:27:02.230474 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.230449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"a1f2f5b3fa6baca7429bbdeea14ceb6ea0a47a16f296d894f3e54bf26d97b34c"}
Jul 09 15:27:02.230790 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.230743    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:02.230911 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.230842    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:02.230969 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.230923    2578 scope.go:117] "RemoveContainer" containerID="29783c0a2d6a33cfedf6f79690685e5581ceb4487b0141dfa8a80464fc3e8df1"
Jul 09 15:27:02.232404 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.232379    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="ee805f93b87c5ef10ca08965de053516c56587914fc53d6e3872ac65920432a4" exitCode=0
Jul 09 15:27:02.232510 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.232403    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"ee805f93b87c5ef10ca08965de053516c56587914fc53d6e3872ac65920432a4"}
Jul 09 15:27:02.247979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:02.247965    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:03.099759 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.099711    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:03.100018 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:03.099845    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:03.239609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.239581    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:27:03.239995 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.239964    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" event={"ID":"de5803a7-2750-43cb-89bd-4fea94ea4898","Type":"ContainerStarted","Data":"b19d8b77e6e662ea9bac8353f119f242a1c3044dffa4fb9b6caf77fb3d8e86d6"}
Jul 09 15:27:03.240348 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.240316    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:03.257180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.257159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:03.299024 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:03.298970    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf" podStartSLOduration=10.060681919 podStartE2EDuration="29.29894908s" totalImagesPullingTime="19.238267161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:03.282940154 +0000 UTC m=+29.743431150" watchObservedRunningTime="2026-07-09 15:27:03.29894908 +0000 UTC m=+29.759440078"
Jul 09 15:27:04.099959 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:04.099929    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:04.100101 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:04.100039    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:04.100101 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:04.100095    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:04.100487 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:04.100191    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:04.245633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:04.245598    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="51fd93ae2f5331b9d7669b13019d433c26c58018e206983ee12c4956d143406a" exitCode=0
Jul 09 15:27:04.245795 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:04.245670    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"51fd93ae2f5331b9d7669b13019d433c26c58018e206983ee12c4956d143406a"}
Jul 09 15:27:05.099174 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:05.099142    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:05.099337 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:05.099253    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:06.099054 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:06.099026    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:06.099478 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:06.099136    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:06.099478 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:06.099190    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:06.099478 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:06.099273    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:06.252579 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:06.252548    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="11159f236df77fd376352790d542ad92fae52f35a1e94bd49969d20d4ba9d10a" exitCode=0
Jul 09 15:27:06.252729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:06.252599    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"11159f236df77fd376352790d542ad92fae52f35a1e94bd49969d20d4ba9d10a"}
Jul 09 15:27:07.099731 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.099703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:07.100195 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.099827    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:07.754543 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.754510    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vg5h6"]
Jul 09 15:27:07.754709 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.754620    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:07.754793 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.754720    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:07.755402 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.755379    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h42wx"]
Jul 09 15:27:07.755534 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.755479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:07.755865 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.755821    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:07.756219 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.756195    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9ktpc"]
Jul 09 15:27:07.756321 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.756307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:07.756464 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.756440    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:07.790144 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.790074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:07.790144 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.790135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:07.790300 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.790225    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 15:27:07.790300 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.790224    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:27:07.790300 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.790273    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret podName:6660f188-abdc-44da-8487-cadab2eb96e7 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:39.790257445 +0000 UTC m=+66.250748420 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret") pod "global-pull-secret-syncer-h42wx" (UID: "6660f188-abdc-44da-8487-cadab2eb96e7") : object "kube-system"/"original-pull-secret" not registered
Jul 09 15:27:07.790300 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.790287    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs podName:3d446ecc-426d-465e-96b6-d42360901d7f nodeName:}" failed. No retries permitted until 2026-07-09 15:27:39.79028013 +0000 UTC m=+66.250771105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs") pod "network-metrics-daemon-9ktpc" (UID: "3d446ecc-426d-465e-96b6-d42360901d7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 15:27:07.891023 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:07.890980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:07.891189 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.891140    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 15:27:07.891189 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.891166    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 15:27:07.891189 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.891178    2578 projected.go:196] Error preparing data for projected volume kube-api-access-frfcr for pod openshift-network-diagnostics/network-check-target-vg5h6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:27:07.891334 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:07.891236    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr podName:734914ef-7a03-4867-b5f1-a6cbf0e26d78 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:39.891218249 +0000 UTC m=+66.351709241 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-frfcr" (UniqueName: "kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr") pod "network-check-target-vg5h6" (UID: "734914ef-7a03-4867-b5f1-a6cbf0e26d78") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 15:27:09.099352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:09.099322    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:09.100079 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:09.099435    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:10.102350 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:10.102319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:10.102783 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:10.102363    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:10.102783 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:10.102479    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9ktpc" podUID="3d446ecc-426d-465e-96b6-d42360901d7f"
Jul 09 15:27:10.102783 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:10.102641    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vg5h6" podUID="734914ef-7a03-4867-b5f1-a6cbf0e26d78"
Jul 09 15:27:11.099680 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.099651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:11.099858 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:11.099740    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h42wx" podUID="6660f188-abdc-44da-8487-cadab2eb96e7"
Jul 09 15:27:11.266019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.265983    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="70d0aba828f8aae16cba563cbcc1d1e52211e4558ea871151dbd62540a439c1e" exitCode=0
Jul 09 15:27:11.266382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.266043    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"70d0aba828f8aae16cba563cbcc1d1e52211e4558ea871151dbd62540a439c1e"}
Jul 09 15:27:11.878859 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.878834    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-248.ec2.internal" event="NodeReady"
Jul 09 15:27:11.878997 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.878955    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 15:27:11.937350 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.937188    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-thqtm"]
Jul 09 15:27:11.939200 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.939181    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r7dgw"]
Jul 09 15:27:11.939312 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.939295    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:11.940963 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.940947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:11.943346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.943329    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 15:27:11.944561 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.944546    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 15:27:11.944671 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.944615    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xw2vr\""
Jul 09 15:27:11.944794 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.944778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 15:27:11.944907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.944892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 15:27:11.945082 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.945065    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 15:27:11.945158 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.945081    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4lm66\""
Jul 09 15:27:11.945603 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.945589    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 15:27:11.950801 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.950776    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r7dgw"]
Jul 09 15:27:11.952767 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:11.952732    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thqtm"]
Jul 09 15:27:12.019449 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-config-volume\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.019599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019454    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-tmp-dir\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.019599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019478    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xlb6\" (UniqueName: \"kubernetes.io/projected/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-kube-api-access-5xlb6\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.019599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019536    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxfck\" (UniqueName: \"kubernetes.io/projected/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-api-access-jxfck\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.019746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019628    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.019746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2fb8f942-6acc-436c-aed4-baad18db12ae-crio-socket\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.019746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb8f942-6acc-436c-aed4-baad18db12ae-data-volume\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.019746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019690    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2fb8f942-6acc-436c-aed4-baad18db12ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.019746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.019714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-metrics-tls\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.037004 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.036978    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x697l"]
Jul 09 15:27:12.038785 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.038729    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.041664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.041644    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 15:27:12.041801 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.041695    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-qhmnt\""
Jul 09 15:27:12.041872 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.041798    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 15:27:12.041995 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.041981    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 15:27:12.049137 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.049113    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x697l"]
Jul 09 15:27:12.102061 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.102040    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:12.102190 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.102044    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:12.104902 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.104883    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 15:27:12.105023 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.104922    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5zg4p\""
Jul 09 15:27:12.105023 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.104966    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 15:27:12.105103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.105056    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krl5b\""
Jul 09 15:27:12.105210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.105188    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 15:27:12.120242 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120225    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-config-volume\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120310 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-tmp-dir\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120310 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120268    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2453b98e-735e-47cf-927b-d3f2ee1661d2-cert\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.120310 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120287    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlb6\" (UniqueName: \"kubernetes.io/projected/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-kube-api-access-5xlb6\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120414 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxfck\" (UniqueName: \"kubernetes.io/projected/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-api-access-jxfck\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120414 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ln6jv\" (UniqueName: \"kubernetes.io/projected/2453b98e-735e-47cf-927b-d3f2ee1661d2-kube-api-access-ln6jv\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.120488 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120536 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2fb8f942-6acc-436c-aed4-baad18db12ae-crio-socket\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120583 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120553    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb8f942-6acc-436c-aed4-baad18db12ae-data-volume\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120583 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2fb8f942-6acc-436c-aed4-baad18db12ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120682 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-metrics-tls\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120682 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-tmp-dir\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120826 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2fb8f942-6acc-436c-aed4-baad18db12ae-crio-socket\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120934 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2fb8f942-6acc-436c-aed4-baad18db12ae-data-volume\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.120968 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-config-volume\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.120968 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.120947    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.124116 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.124088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2fb8f942-6acc-436c-aed4-baad18db12ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.124207 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.124162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-metrics-tls\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.132129 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.132103    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxfck\" (UniqueName: \"kubernetes.io/projected/2fb8f942-6acc-436c-aed4-baad18db12ae-kube-api-access-jxfck\") pod \"insights-runtime-extractor-r7dgw\" (UID: \"2fb8f942-6acc-436c-aed4-baad18db12ae\") " pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.132209 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.132109    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlb6\" (UniqueName: \"kubernetes.io/projected/de21f5ac-bf48-45f1-b93c-696a5e9b8dec-kube-api-access-5xlb6\") pod \"dns-default-thqtm\" (UID: \"de21f5ac-bf48-45f1-b93c-696a5e9b8dec\") " pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.221331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.221299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ln6jv\" (UniqueName: \"kubernetes.io/projected/2453b98e-735e-47cf-927b-d3f2ee1661d2-kube-api-access-ln6jv\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.221474 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.221364    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2453b98e-735e-47cf-927b-d3f2ee1661d2-cert\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.223746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.223727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2453b98e-735e-47cf-927b-d3f2ee1661d2-cert\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.231486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.231467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ln6jv\" (UniqueName: \"kubernetes.io/projected/2453b98e-735e-47cf-927b-d3f2ee1661d2-kube-api-access-ln6jv\") pod \"ingress-canary-x697l\" (UID: \"2453b98e-735e-47cf-927b-d3f2ee1661d2\") " pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.252180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.252157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:12.257440 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.257419    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r7dgw"
Jul 09 15:27:12.273222 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.273141    2578 generic.go:356] "Generic (PLEG): container finished" podID="66e56408-1d75-4c3b-ac75-bf0c62ee4f07" containerID="fe14df7e8fcae2eab0d0d90ffbae7264e699dd8367a769d359433ffeccd2f3cb" exitCode=0
Jul 09 15:27:12.273222 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.273183    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerDied","Data":"fe14df7e8fcae2eab0d0d90ffbae7264e699dd8367a769d359433ffeccd2f3cb"}
Jul 09 15:27:12.347029 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.346984    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x697l"
Jul 09 15:27:12.415549 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.415388    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r7dgw"]
Jul 09 15:27:12.420021 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:12.419553    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fb8f942_6acc_436c_aed4_baad18db12ae.slice/crio-dacd8f62e00baf51abd42448f99cc33405dda512df7ce618a39840c7000da45d WatchSource:0}: Error finding container dacd8f62e00baf51abd42448f99cc33405dda512df7ce618a39840c7000da45d: Status 404 returned error can't find the container with id dacd8f62e00baf51abd42448f99cc33405dda512df7ce618a39840c7000da45d
Jul 09 15:27:12.420663 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.420630    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thqtm"]
Jul 09 15:27:12.423011 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:12.422984    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde21f5ac_bf48_45f1_b93c_696a5e9b8dec.slice/crio-050293bebbe28e3b399c2d2ca3fc086e39dff6c8c72dbd2774f81aadc4439411 WatchSource:0}: Error finding container 050293bebbe28e3b399c2d2ca3fc086e39dff6c8c72dbd2774f81aadc4439411: Status 404 returned error can't find the container with id 050293bebbe28e3b399c2d2ca3fc086e39dff6c8c72dbd2774f81aadc4439411
Jul 09 15:27:12.481024 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.480998    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x697l"]
Jul 09 15:27:12.483667 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:12.483643    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2453b98e_735e_47cf_927b_d3f2ee1661d2.slice/crio-f65d784722b9052a3d3dbeada9be57d1780cf981409e1a14780637f353dc43a5 WatchSource:0}: Error finding container f65d784722b9052a3d3dbeada9be57d1780cf981409e1a14780637f353dc43a5: Status 404 returned error can't find the container with id f65d784722b9052a3d3dbeada9be57d1780cf981409e1a14780637f353dc43a5
Jul 09 15:27:12.597402 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.597338    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:27:12.603546 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.603524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.606446 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 09 15:27:12.606548 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2qtgs\""
Jul 09 15:27:12.606626 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606580    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 09 15:27:12.606687 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606660    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 09 15:27:12.606737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606729    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 09 15:27:12.606960 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606944    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 15:27:12.607028 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606968    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 15:27:12.607028 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.606994    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 09 15:27:12.615531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.615508    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:27:12.726905 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.726877    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.727042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.726943    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.727042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.726989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.727042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.727007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9894\" (UniqueName: \"kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.727139 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.727046    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.727139 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.727066    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828193 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828313 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828217    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828313 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828313 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9894\" (UniqueName: \"kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828414 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828418    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.828971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.828946    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.831669 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.831652    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.831730 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.831673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.838021 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.838003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9894\" (UniqueName: \"kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894\") pod \"console-688cfd7ff7-qdnqs\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") " pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:12.915392 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:12.915326    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:13.099886 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.099824    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:27:13.100623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.100108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:13.103420 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:13.103387    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf1f9fea_bcdc_44aa_898f_d4e20ba468dd.slice/crio-a2f800d5c654c0b7a55f8778eabde42e88adf2d5bfa52b81faa50e32ef0999a1 WatchSource:0}: Error finding container a2f800d5c654c0b7a55f8778eabde42e88adf2d5bfa52b81faa50e32ef0999a1: Status 404 returned error can't find the container with id a2f800d5c654c0b7a55f8778eabde42e88adf2d5bfa52b81faa50e32ef0999a1
Jul 09 15:27:13.104531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.104367    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 15:27:13.278795 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.278739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688cfd7ff7-qdnqs" event={"ID":"af1f9fea-bcdc-44aa-898f-d4e20ba468dd","Type":"ContainerStarted","Data":"a2f800d5c654c0b7a55f8778eabde42e88adf2d5bfa52b81faa50e32ef0999a1"}
Jul 09 15:27:13.280302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.280273    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x697l" event={"ID":"2453b98e-735e-47cf-927b-d3f2ee1661d2","Type":"ContainerStarted","Data":"f65d784722b9052a3d3dbeada9be57d1780cf981409e1a14780637f353dc43a5"}
Jul 09 15:27:13.284389 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.284361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" event={"ID":"66e56408-1d75-4c3b-ac75-bf0c62ee4f07","Type":"ContainerStarted","Data":"28878114d152a7c51e7ff314f542a77320d937873e9f29c774cb0dd36f959d25"}
Jul 09 15:27:13.286821 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.286797    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r7dgw" event={"ID":"2fb8f942-6acc-436c-aed4-baad18db12ae","Type":"ContainerStarted","Data":"dacd8f62e00baf51abd42448f99cc33405dda512df7ce618a39840c7000da45d"}
Jul 09 15:27:13.288145 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.288122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thqtm" event={"ID":"de21f5ac-bf48-45f1-b93c-696a5e9b8dec","Type":"ContainerStarted","Data":"050293bebbe28e3b399c2d2ca3fc086e39dff6c8c72dbd2774f81aadc4439411"}
Jul 09 15:27:13.315537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:13.315487    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r9sjn" podStartSLOduration=8.878532472 podStartE2EDuration="39.315466604s" totalImagesPullingTime="30.436934132s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:13.311567612 +0000 UTC m=+39.772058624" watchObservedRunningTime="2026-07-09 15:27:13.315466604 +0000 UTC m=+39.775957600"
Jul 09 15:27:18.636836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.636803    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rkfw9"]
Jul 09 15:27:18.692186 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.692153    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rkfw9"]
Jul 09 15:27:18.692352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.692271    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.696191 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696159    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 09 15:27:18.696191 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696163    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 09 15:27:18.696386 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696192    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 15:27:18.696386 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696210    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mgtnr\""
Jul 09 15:27:18.696386 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696215    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 15:27:18.696740 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.696710    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 15:27:18.787820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.787790    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.787965 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.787823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.787965 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.787842    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dda39670-c931-41e8-a6b8-e60ad725f98f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.787965 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.787923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zd49h\" (UniqueName: \"kubernetes.io/projected/dda39670-c931-41e8-a6b8-e60ad725f98f-kube-api-access-zd49h\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.889042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.888980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zd49h\" (UniqueName: \"kubernetes.io/projected/dda39670-c931-41e8-a6b8-e60ad725f98f-kube-api-access-zd49h\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.889187 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.889047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.889187 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:18.889131    2578 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 09 15:27:18.889187 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.889168    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.889187 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:18.889182    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls podName:dda39670-c931-41e8-a6b8-e60ad725f98f nodeName:}" failed. No retries permitted until 2026-07-09 15:27:19.389167663 +0000 UTC m=+45.849658642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls") pod "prometheus-operator-64785b6675-rkfw9" (UID: "dda39670-c931-41e8-a6b8-e60ad725f98f") : secret "prometheus-operator-tls" not found
Jul 09 15:27:18.889382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.889207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dda39670-c931-41e8-a6b8-e60ad725f98f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.889788 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.889773    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dda39670-c931-41e8-a6b8-e60ad725f98f-metrics-client-ca\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.891954 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.891930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:18.902555 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:18.902532    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zd49h\" (UniqueName: \"kubernetes.io/projected/dda39670-c931-41e8-a6b8-e60ad725f98f-kube-api-access-zd49h\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:19.393138 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:19.393100    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:19.395977 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:19.395950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/dda39670-c931-41e8-a6b8-e60ad725f98f-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-rkfw9\" (UID: \"dda39670-c931-41e8-a6b8-e60ad725f98f\") " pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:19.601922 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:19.601888    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9"
Jul 09 15:27:20.189219 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.189190    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:27:20.206092 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.206057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.215524 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.212792    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:27:20.232932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.231953    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 09 15:27:20.302249 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.302225    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-rkfw9"]
Jul 09 15:27:20.304511 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:20.304482    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda39670_c931_41e8_a6b8_e60ad725f98f.slice/crio-935242f384daddbd4527ffb73991706cb9c9fde080389f17217be0caad123b7b WatchSource:0}: Error finding container 935242f384daddbd4527ffb73991706cb9c9fde080389f17217be0caad123b7b: Status 404 returned error can't find the container with id 935242f384daddbd4527ffb73991706cb9c9fde080389f17217be0caad123b7b
Jul 09 15:27:20.400877 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.400854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.400990 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.400898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.400990 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.400934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.400990 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.400960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw75p\" (UniqueName: \"kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.401170 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.401072    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.401170 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.401140    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.401268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.401178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501797 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501766    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501797 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501999 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501999 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501999 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501942    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.501999 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.501980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.502234 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.502009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw75p\" (UniqueName: \"kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.502669 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.502589    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.502784 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.502688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.502871 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.502800    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.503171 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.503147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.504651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.504620    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.504880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.504860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.511737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.511715    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw75p\" (UniqueName: \"kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p\") pod \"console-6c6889fdf8-qtprh\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") " pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.547127 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.547107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:20.691410 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:20.691275    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:27:20.693851 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:20.693827    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4314040c_c39c_48df_a337_a24de545376c.slice/crio-7527ce5bdafd0e0d8da1760ac53ecc99e187274747f4335c6b417b20f505b5b3 WatchSource:0}: Error finding container 7527ce5bdafd0e0d8da1760ac53ecc99e187274747f4335c6b417b20f505b5b3: Status 404 returned error can't find the container with id 7527ce5bdafd0e0d8da1760ac53ecc99e187274747f4335c6b417b20f505b5b3
Jul 09 15:27:21.313692 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.313653    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r7dgw" event={"ID":"2fb8f942-6acc-436c-aed4-baad18db12ae","Type":"ContainerStarted","Data":"ebfe190ee0568d3696289c4c0d222ec5fa6ddd960207025c550f5edcbb8cbd93"}
Jul 09 15:27:21.315725 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.315694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thqtm" event={"ID":"de21f5ac-bf48-45f1-b93c-696a5e9b8dec","Type":"ContainerStarted","Data":"08ae35b748852a0602c4b6d3aaafc59bb64c30c5d3518717f3fb22b4a88b7aec"}
Jul 09 15:27:21.315879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.315770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thqtm" event={"ID":"de21f5ac-bf48-45f1-b93c-696a5e9b8dec","Type":"ContainerStarted","Data":"e3d7c62fa5970eed5cfa747671c9d6a520f1977f79206fcb7767db977d58536c"}
Jul 09 15:27:21.315879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.315802    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:21.317372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.317344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9" event={"ID":"dda39670-c931-41e8-a6b8-e60ad725f98f","Type":"ContainerStarted","Data":"935242f384daddbd4527ffb73991706cb9c9fde080389f17217be0caad123b7b"}
Jul 09 15:27:21.319042 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.319013    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688cfd7ff7-qdnqs" event={"ID":"af1f9fea-bcdc-44aa-898f-d4e20ba468dd","Type":"ContainerStarted","Data":"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"}
Jul 09 15:27:21.320978 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.320926    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x697l" event={"ID":"2453b98e-735e-47cf-927b-d3f2ee1661d2","Type":"ContainerStarted","Data":"7cf28b0302a1b4398b9dd68903127840c1f72638d832e024c760323d38bb0e29"}
Jul 09 15:27:21.322547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.322528    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6889fdf8-qtprh" event={"ID":"4314040c-c39c-48df-a337-a24de545376c","Type":"ContainerStarted","Data":"011adf8cb92e0e1c300f2252593790aa7d3a68d366c4a8ba6a355fa55a5a4a90"}
Jul 09 15:27:21.322625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.322560    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6889fdf8-qtprh" event={"ID":"4314040c-c39c-48df-a337-a24de545376c","Type":"ContainerStarted","Data":"7527ce5bdafd0e0d8da1760ac53ecc99e187274747f4335c6b417b20f505b5b3"}
Jul 09 15:27:21.338552 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.338502    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-thqtm" podStartSLOduration=2.603323391 podStartE2EDuration="10.338486874s" totalImagesPullingTime="7.735163483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:21.338315872 +0000 UTC m=+47.798806871" watchObservedRunningTime="2026-07-09 15:27:21.338486874 +0000 UTC m=+47.798977874"
Jul 09 15:27:21.359747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.359701    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c6889fdf8-qtprh" podStartSLOduration=1.359687293 podStartE2EDuration="1.359687293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:21.359302364 +0000 UTC m=+47.819793360" watchObservedRunningTime="2026-07-09 15:27:21.359687293 +0000 UTC m=+47.820178268"
Jul 09 15:27:21.380564 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.380517    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688cfd7ff7-qdnqs" podStartSLOduration=2.326030053 podStartE2EDuration="9.380508036s" totalImagesPullingTime="7.054477983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:21.37978757 +0000 UTC m=+47.840278566" watchObservedRunningTime="2026-07-09 15:27:21.380508036 +0000 UTC m=+47.840999011"
Jul 09 15:27:21.396103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:21.396051    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x697l" podStartSLOduration=1.721281659 podStartE2EDuration="9.396036657s" totalImagesPullingTime="7.674754998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:21.395814984 +0000 UTC m=+47.856305982" watchObservedRunningTime="2026-07-09 15:27:21.396036657 +0000 UTC m=+47.856527635"
Jul 09 15:27:22.915959 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:22.915924    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:22.916357 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:22.916022    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:22.921006 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:22.920983    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:23.332164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:23.332124    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r7dgw" event={"ID":"2fb8f942-6acc-436c-aed4-baad18db12ae","Type":"ContainerStarted","Data":"c3f3d1c6eb2edf80edd9765d77e7f6aecb68189c13df8b20cfec68685202fd4c"}
Jul 09 15:27:23.338344 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:23.338314    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9" event={"ID":"dda39670-c931-41e8-a6b8-e60ad725f98f","Type":"ContainerStarted","Data":"5f0d4d9f1a8afca1e9fda12b7acae5e9b04589112ca7da4435d51ac143aa8bbb"}
Jul 09 15:27:23.338498 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:23.338369    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9" event={"ID":"dda39670-c931-41e8-a6b8-e60ad725f98f","Type":"ContainerStarted","Data":"f2de712a36aa251d089acc2f0785ee632a2697c6a9f07c65720a10708bf8f3fd"}
Jul 09 15:27:23.343475 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:23.343450    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:27:23.381627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:23.381587    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-rkfw9" podStartSLOduration=3.405213288 podStartE2EDuration="5.381572106s" totalImagesPullingTime="1.976358818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:23.359690504 +0000 UTC m=+49.820181498" watchObservedRunningTime="2026-07-09 15:27:23.381572106 +0000 UTC m=+49.842063101"
Jul 09 15:27:25.053606 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.053574    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"]
Jul 09 15:27:25.097505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.097472    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cm86b"]
Jul 09 15:27:25.097677 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.097585    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.100269 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.100240    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 15:27:25.100431 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.100283    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-rg46g\""
Jul 09 15:27:25.100431 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.100252    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 15:27:25.116881 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.116860    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"]
Jul 09 15:27:25.117021 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.116908    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mf8sj"]
Jul 09 15:27:25.117021 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.117006    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.119717 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.119696    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 15:27:25.120011 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.119990    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 15:27:25.120316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.120285    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-ln4zq\""
Jul 09 15:27:25.120654 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.120615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 15:27:25.140623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.140746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdbzs\" (UniqueName: \"kubernetes.io/projected/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-kube-api-access-jdbzs\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.140746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.140845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140775    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.140845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.140962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.140962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140901    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cm86b"]
Jul 09 15:27:25.140962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140910    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hfnl\" (UniqueName: \"kubernetes.io/projected/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-api-access-5hfnl\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.140962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140941    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.141118 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140967    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.141118 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.140984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc763105-2ba6-45fa-9585-76f09bd4d94d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.141118 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.141002    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.144883 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.144865    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 15:27:25.145315 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.145286    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 15:27:25.150087 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.150020    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-frh6w\""
Jul 09 15:27:25.150186 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.150108    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 15:27:25.242230 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-textfile\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242230 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-wtmp\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc763105-2ba6-45fa-9585-76f09bd4d94d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.242450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242373    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.242450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbzs\" (UniqueName: \"kubernetes.io/projected/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-kube-api-access-jdbzs\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.242450 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242436    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s7fr\" (UniqueName: \"kubernetes.io/projected/017c6d42-a371-44b4-835f-6337ab0bb98a-kube-api-access-5s7fr\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242476    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.242661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242543    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-root\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-metrics-client-ca\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc763105-2ba6-45fa-9585-76f09bd4d94d-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.242661 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242680    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-sys\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:25.242654    2578 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-tls\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:25.242785    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls podName:a268dbc2-70ec-4b03-b96f-6caf5dbb9540 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:25.742745346 +0000 UTC m=+52.203236324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-r265z" (UID: "a268dbc2-70ec-4b03-b96f-6caf5dbb9540") : secret "openshift-state-metrics-tls" not found
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.242931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242906    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hfnl\" (UniqueName: \"kubernetes.io/projected/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-api-access-5hfnl\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.243320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.243320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.242981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.243320 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:25.243065    2578 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 09 15:27:25.243320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.243090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.243320 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:25.243120    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls podName:fc763105-2ba6-45fa-9585-76f09bd4d94d nodeName:}" failed. No retries permitted until 2026-07-09 15:27:25.743106342 +0000 UTC m=+52.203597324 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-cm86b" (UID: "fc763105-2ba6-45fa-9585-76f09bd4d94d") : secret "kube-state-metrics-tls" not found
Jul 09 15:27:25.243724 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.243461    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.243724 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.243568    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.245308 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.245281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.245394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.245378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.254057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.254031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbzs\" (UniqueName: \"kubernetes.io/projected/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-kube-api-access-jdbzs\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.254451 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.254434    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hfnl\" (UniqueName: \"kubernetes.io/projected/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-api-access-5hfnl\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.344309 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344257    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-root\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344309 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344301    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-metrics-client-ca\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344319    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-sys\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344337    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-tls\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344371    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-root\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344392    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-sys\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-textfile\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344461    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-wtmp\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.344625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s7fr\" (UniqueName: \"kubernetes.io/projected/017c6d42-a371-44b4-835f-6337ab0bb98a-kube-api-access-5s7fr\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.345063 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344664    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-wtmp\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.345063 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-textfile\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.345063 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.344961    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-accelerators-collector-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.345268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.345236    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/017c6d42-a371-44b4-835f-6337ab0bb98a-metrics-client-ca\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.347173 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.347131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-tls\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.347609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.347587    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/017c6d42-a371-44b4-835f-6337ab0bb98a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.356626 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.356608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s7fr\" (UniqueName: \"kubernetes.io/projected/017c6d42-a371-44b4-835f-6337ab0bb98a-kube-api-access-5s7fr\") pod \"node-exporter-mf8sj\" (UID: \"017c6d42-a371-44b4-835f-6337ab0bb98a\") " pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.451176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.451151    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mf8sj"
Jul 09 15:27:25.663863 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:25.663832    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017c6d42_a371_44b4_835f_6337ab0bb98a.slice/crio-6e575f7fc836f03dd1fcb2c5edd6c438f331f59f1ac1634293ce46db34d2584c WatchSource:0}: Error finding container 6e575f7fc836f03dd1fcb2c5edd6c438f331f59f1ac1634293ce46db34d2584c: Status 404 returned error can't find the container with id 6e575f7fc836f03dd1fcb2c5edd6c438f331f59f1ac1634293ce46db34d2584c
Jul 09 15:27:25.748695 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.748670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:25.748899 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.748740    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.751062 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.751038    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc763105-2ba6-45fa-9585-76f09bd4d94d-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-cm86b\" (UID: \"fc763105-2ba6-45fa-9585-76f09bd4d94d\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:25.751158 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:25.751040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/a268dbc2-70ec-4b03-b96f-6caf5dbb9540-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-r265z\" (UID: \"a268dbc2-70ec-4b03-b96f-6caf5dbb9540\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:26.008764 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.008677    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"
Jul 09 15:27:26.026949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.026929    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b"
Jul 09 15:27:26.129734 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.129703    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:27:26.152592 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.150287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.152592 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.151866    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:27:26.155111 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.155086    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7chhh\""
Jul 09 15:27:26.155402 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.155386    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 15:27:26.155503 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.155407    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 15:27:26.155648 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.155632    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 15:27:26.155738 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.155725    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 15:27:26.156066 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.156050    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 15:27:26.156261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.156245    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 15:27:26.156462 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.156446    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 15:27:26.156633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.156619    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 15:27:26.156979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.156808    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 15:27:26.190478 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.190451    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-r265z"]
Jul 09 15:27:26.191775 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:26.191728    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda268dbc2_70ec_4b03_b96f_6caf5dbb9540.slice/crio-a447c65755a005199cad861c87bbf10c1af5049962e4b17f14f087d5f9cc3e5d WatchSource:0}: Error finding container a447c65755a005199cad861c87bbf10c1af5049962e4b17f14f087d5f9cc3e5d: Status 404 returned error can't find the container with id a447c65755a005199cad861c87bbf10c1af5049962e4b17f14f087d5f9cc3e5d
Jul 09 15:27:26.233479 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.233453    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-cm86b"]
Jul 09 15:27:26.239899 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:26.239872    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc763105_2ba6_45fa_9585_76f09bd4d94d.slice/crio-c1f50d4625f71030150b258253d645be4a6f8fdd39aad01705b2dde5f969b532 WatchSource:0}: Error finding container c1f50d4625f71030150b258253d645be4a6f8fdd39aad01705b2dde5f969b532: Status 404 returned error can't find the container with id c1f50d4625f71030150b258253d645be4a6f8fdd39aad01705b2dde5f969b532
Jul 09 15:27:26.252341 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252315    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252446 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252362    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252446 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252459    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252509    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252637 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrpvr\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252689 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252698    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252854 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252741    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252854 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252833    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252947 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.252947 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.252905    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353339 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353439 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353347    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrpvr\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353439 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353580 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353580 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353580 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353580 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353588    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353667    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.353791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.353768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.354093 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.354069    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.354147 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.354111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.355145 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.354697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mf8sj" event={"ID":"017c6d42-a371-44b4-835f-6337ab0bb98a","Type":"ContainerStarted","Data":"6e575f7fc836f03dd1fcb2c5edd6c438f331f59f1ac1634293ce46db34d2584c"}
Jul 09 15:27:26.358180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.357166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.358180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.357377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.358180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.357384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.358180 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.358137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.358416 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.358286    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.359880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.359857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.360409 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.359859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.360409 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.360218    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.361962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.361936    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.362321 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.362297    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.362808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.362774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z" event={"ID":"a268dbc2-70ec-4b03-b96f-6caf5dbb9540","Type":"ContainerStarted","Data":"5b81ae3a755f5a60c06c91c96bde9f72bfb97fd62c428494d3e6e807bf6645bf"}
Jul 09 15:27:26.362900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.362841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z" event={"ID":"a268dbc2-70ec-4b03-b96f-6caf5dbb9540","Type":"ContainerStarted","Data":"a447c65755a005199cad861c87bbf10c1af5049962e4b17f14f087d5f9cc3e5d"}
Jul 09 15:27:26.365192 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.365169    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrpvr\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr\") pod \"alertmanager-main-0\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.365493 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.365469    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r7dgw" event={"ID":"2fb8f942-6acc-436c-aed4-baad18db12ae","Type":"ContainerStarted","Data":"7452f6d12f5bce1978cc20590b2d504d56b3f799ec796b7971ddb7c08a61e1bb"}
Jul 09 15:27:26.366924 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.366837    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b" event={"ID":"fc763105-2ba6-45fa-9585-76f09bd4d94d","Type":"ContainerStarted","Data":"c1f50d4625f71030150b258253d645be4a6f8fdd39aad01705b2dde5f969b532"}
Jul 09 15:27:26.463870 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.463835    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:27:26.631122 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.631032    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r7dgw" podStartSLOduration=2.576959934 podStartE2EDuration="15.630997291s" totalImagesPullingTime="13.054037357s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:27:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:26.406903908 +0000 UTC m=+52.867394907" watchObservedRunningTime="2026-07-09 15:27:26.630997291 +0000 UTC m=+53.091488285"
Jul 09 15:27:26.632400 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:26.632361    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:27:26.807881 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:26.807816    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode027b13f_1d3f_40a5_a38e_6269063ea80a.slice/crio-010855fc888fadfe5edeb888eede36ce11de34ec09dfe6b65e776459361e3358 WatchSource:0}: Error finding container 010855fc888fadfe5edeb888eede36ce11de34ec09dfe6b65e776459361e3358: Status 404 returned error can't find the container with id 010855fc888fadfe5edeb888eede36ce11de34ec09dfe6b65e776459361e3358
Jul 09 15:27:27.046739 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.046708    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-684ff48bb7-l946t"]
Jul 09 15:27:27.068209 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.068148    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.069578 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.069553    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-684ff48bb7-l946t"]
Jul 09 15:27:27.071487 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.071467    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 09 15:27:27.071848 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.071824    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 09 15:27:27.071950 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.071880    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ssgpc\""
Jul 09 15:27:27.072117 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.072099    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 09 15:27:27.072198 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.072182    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 09 15:27:27.072252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.072227    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-299agbqgj486d\""
Jul 09 15:27:27.073385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.073365    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 09 15:27:27.160205 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-grpc-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160250    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3abd3de-08f5-42d9-a560-19690bbccd3c-metrics-client-ca\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4fps\" (UniqueName: \"kubernetes.io/projected/f3abd3de-08f5-42d9-a560-19690bbccd3c-kube-api-access-g4fps\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160483    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.160597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.160525    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261398 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4fps\" (UniqueName: \"kubernetes.io/projected/f3abd3de-08f5-42d9-a560-19690bbccd3c-kube-api-access-g4fps\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261421    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261782 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261672    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261782 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-grpc-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.261907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.261901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3abd3de-08f5-42d9-a560-19690bbccd3c-metrics-client-ca\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.262966 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.262737    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3abd3de-08f5-42d9-a560-19690bbccd3c-metrics-client-ca\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.264680 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.264657    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.264801 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.264705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.264801 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.264709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.266250 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.266225    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.266331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.266228    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.266331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.266270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f3abd3de-08f5-42d9-a560-19690bbccd3c-secret-grpc-tls\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.272401 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.272372    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4fps\" (UniqueName: \"kubernetes.io/projected/f3abd3de-08f5-42d9-a560-19690bbccd3c-kube-api-access-g4fps\") pod \"thanos-querier-684ff48bb7-l946t\" (UID: \"f3abd3de-08f5-42d9-a560-19690bbccd3c\") " pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.376339 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.375683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mf8sj" event={"ID":"017c6d42-a371-44b4-835f-6337ab0bb98a","Type":"ContainerStarted","Data":"06272d508090e517d9ee40e3d2632c93e01ed87c9e7211d9bcdbb0eb5e67b176"}
Jul 09 15:27:27.378078 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.378052    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:27.379304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.379165    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z" event={"ID":"a268dbc2-70ec-4b03-b96f-6caf5dbb9540","Type":"ContainerStarted","Data":"b1ae65b9f0119a961600b52e417e90010f53c2ef57f7406850d337d621e7a14d"}
Jul 09 15:27:27.382902 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.382607    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"010855fc888fadfe5edeb888eede36ce11de34ec09dfe6b65e776459361e3358"}
Jul 09 15:27:27.551320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:27.551288    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-684ff48bb7-l946t"]
Jul 09 15:27:27.554869 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:27.554841    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3abd3de_08f5_42d9_a560_19690bbccd3c.slice/crio-a820c523bdda98d0e464cd27795d282501ec8e7b15a55f4252aa01b66e0c31cc WatchSource:0}: Error finding container a820c523bdda98d0e464cd27795d282501ec8e7b15a55f4252aa01b66e0c31cc: Status 404 returned error can't find the container with id a820c523bdda98d0e464cd27795d282501ec8e7b15a55f4252aa01b66e0c31cc
Jul 09 15:27:28.387503 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:28.387467    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"a820c523bdda98d0e464cd27795d282501ec8e7b15a55f4252aa01b66e0c31cc"}
Jul 09 15:27:28.388800 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:28.388780    2578 generic.go:356] "Generic (PLEG): container finished" podID="017c6d42-a371-44b4-835f-6337ab0bb98a" containerID="06272d508090e517d9ee40e3d2632c93e01ed87c9e7211d9bcdbb0eb5e67b176" exitCode=0
Jul 09 15:27:28.388888 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:28.388816    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mf8sj" event={"ID":"017c6d42-a371-44b4-835f-6337ab0bb98a","Type":"ContainerDied","Data":"06272d508090e517d9ee40e3d2632c93e01ed87c9e7211d9bcdbb0eb5e67b176"}
Jul 09 15:27:29.396642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.396481    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5" exitCode=0
Jul 09 15:27:29.396642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.396561    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5"}
Jul 09 15:27:29.399797 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.399736    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mf8sj" event={"ID":"017c6d42-a371-44b4-835f-6337ab0bb98a","Type":"ContainerStarted","Data":"a0ad2bb0c154047902080e6cb06b1f96166ce826dbd4567ed2672d4da7fc2076"}
Jul 09 15:27:29.402459 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.402281    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z" event={"ID":"a268dbc2-70ec-4b03-b96f-6caf5dbb9540","Type":"ContainerStarted","Data":"7485253f25bedc407867f59f79affaae7fab52b2505b6d8bf7bc45476af31fd5"}
Jul 09 15:27:29.405312 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.405175    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b" event={"ID":"fc763105-2ba6-45fa-9585-76f09bd4d94d","Type":"ContainerStarted","Data":"e7c46f43077d65f5a2887c13e91a35ce80225d0c130f2677e39ef6e18892fac4"}
Jul 09 15:27:29.464101 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.464048    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-r265z" podStartSLOduration=1.8040282680000002 podStartE2EDuration="4.464029043s" totalImagesPullingTime="2.660000775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:29.463237087 +0000 UTC m=+55.923728085" watchObservedRunningTime="2026-07-09 15:27:29.464029043 +0000 UTC m=+55.924520041"
Jul 09 15:27:29.471384 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.471347    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6455bc7799-wh8xf"]
Jul 09 15:27:29.476743 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.476717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.481857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.481080    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 09 15:27:29.481857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.481254    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-e8phuvv05k4if\""
Jul 09 15:27:29.481857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.481273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 09 15:27:29.481857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.481082    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 09 15:27:29.481857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.481457    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x2tnc\""
Jul 09 15:27:29.482509 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.482353    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 15:27:29.494486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.494464    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6455bc7799-wh8xf"]
Jul 09 15:27:29.582879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.582847    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-client-certs\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.582879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.582882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-client-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.583090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.582913    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg8kc\" (UniqueName: \"kubernetes.io/projected/f13c58b3-a475-4cc1-8f9c-65df8b952cda-kube-api-access-qg8kc\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.583090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.582972    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f13c58b3-a475-4cc1-8f9c-65df8b952cda-audit-log\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.583090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.583018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-metrics-server-audit-profiles\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.583090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.583055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-tls\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.583090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.583086    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684254 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684173    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-client-certs\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684254 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-client-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684254 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg8kc\" (UniqueName: \"kubernetes.io/projected/f13c58b3-a475-4cc1-8f9c-65df8b952cda-kube-api-access-qg8kc\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f13c58b3-a475-4cc1-8f9c-65df8b952cda-audit-log\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684317    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-metrics-server-audit-profiles\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-tls\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.684519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.684377    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.685071 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.685043    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/f13c58b3-a475-4cc1-8f9c-65df8b952cda-audit-log\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.685465 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.685402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.685624 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.685598    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/f13c58b3-a475-4cc1-8f9c-65df8b952cda-metrics-server-audit-profiles\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.687372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.687343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-client-certs\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.688321 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.688279    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-client-ca-bundle\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.688703 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.688680    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/f13c58b3-a475-4cc1-8f9c-65df8b952cda-secret-metrics-server-tls\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.695758 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.695727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg8kc\" (UniqueName: \"kubernetes.io/projected/f13c58b3-a475-4cc1-8f9c-65df8b952cda-kube-api-access-qg8kc\") pod \"metrics-server-6455bc7799-wh8xf\" (UID: \"f13c58b3-a475-4cc1-8f9c-65df8b952cda\") " pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.790062 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.790022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:29.824242 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.823959    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"]
Jul 09 15:27:29.826894 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.826862    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:29.830160 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.829691    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 09 15:27:29.830160 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.829974    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-nxsfr\""
Jul 09 15:27:29.839821 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.839796    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"]
Jul 09 15:27:29.886794 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.886737    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjc68\" (UniqueName: \"kubernetes.io/projected/7d058837-d814-40ea-9ef6-82394d96ea51-kube-api-access-cjc68\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:29.886947 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.886860    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:29.958798 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.958692    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6455bc7799-wh8xf"]
Jul 09 15:27:29.961569 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:29.961540    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf13c58b3_a475_4cc1_8f9c_65df8b952cda.slice/crio-64b36e63366102eb877b5eed859d0f50ac441b0260fe6f5dff8a38cc5fe1c638 WatchSource:0}: Error finding container 64b36e63366102eb877b5eed859d0f50ac441b0260fe6f5dff8a38cc5fe1c638: Status 404 returned error can't find the container with id 64b36e63366102eb877b5eed859d0f50ac441b0260fe6f5dff8a38cc5fe1c638
Jul 09 15:27:29.987651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.987624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:29.987821 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.987800    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjc68\" (UniqueName: \"kubernetes.io/projected/7d058837-d814-40ea-9ef6-82394d96ea51-kube-api-access-cjc68\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:29.987887 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:29.987823    2578 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 09 15:27:29.987939 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:27:29.987897    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert podName:7d058837-d814-40ea-9ef6-82394d96ea51 nodeName:}" failed. No retries permitted until 2026-07-09 15:27:30.487876138 +0000 UTC m=+56.948367126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-s5dfq" (UID: "7d058837-d814-40ea-9ef6-82394d96ea51") : secret "monitoring-plugin-cert" not found
Jul 09 15:27:29.999733 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:29.999712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjc68\" (UniqueName: \"kubernetes.io/projected/7d058837-d814-40ea-9ef6-82394d96ea51-kube-api-access-cjc68\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:30.280090 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.279540    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"]
Jul 09 15:27:30.282745 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.282724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.287194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.287171    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 09 15:27:30.287900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.287879    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 09 15:27:30.287900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.287891    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 09 15:27:30.288290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.288121    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 09 15:27:30.288290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.288160    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-w52j5\""
Jul 09 15:27:30.288290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.288131    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 09 15:27:30.297051 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.297030    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 09 15:27:30.297642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.297614    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"]
Jul 09 15:27:30.391163 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-federate-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391163 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391158    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-metrics-client-ca\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391176    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391200    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw2pc\" (UniqueName: \"kubernetes.io/projected/07adf231-3ea6-485b-8bea-5142b1dfeb98-kube-api-access-sw2pc\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391341    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.391390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.391371    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-serving-certs-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.413947 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.413908    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mf8sj" event={"ID":"017c6d42-a371-44b4-835f-6337ab0bb98a","Type":"ContainerStarted","Data":"c46e074a9a640703309079e204b47f55ec081ed5a29cd95b47560f710283c4f2"}
Jul 09 15:27:30.415726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.415678    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf" event={"ID":"f13c58b3-a475-4cc1-8f9c-65df8b952cda","Type":"ContainerStarted","Data":"64b36e63366102eb877b5eed859d0f50ac441b0260fe6f5dff8a38cc5fe1c638"}
Jul 09 15:27:30.419598 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.419571    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b" event={"ID":"fc763105-2ba6-45fa-9585-76f09bd4d94d","Type":"ContainerStarted","Data":"a186ca8161dc8b9a047f0de8f03e4ce915c6bb809ad85a83f2818c0b63556c51"}
Jul 09 15:27:30.419730 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.419628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b" event={"ID":"fc763105-2ba6-45fa-9585-76f09bd4d94d","Type":"ContainerStarted","Data":"a34c92881db780232de29ed28beb27f73570c4ba063f65dc7f42fb818551381a"}
Jul 09 15:27:30.449820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.449772    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mf8sj" podStartSLOduration=3.823310028 podStartE2EDuration="5.449740143s" totalImagesPullingTime="1.494156511s" totalInitContainerRuntime="132.273604ms" isStatefulPod=true podCreationTimestamp="2026-07-09 15:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:30.442970203 +0000 UTC m=+56.903461201" watchObservedRunningTime="2026-07-09 15:27:30.449740143 +0000 UTC m=+56.910231118"
Jul 09 15:27:30.468540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.468499    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-cm86b" podStartSLOduration=2.5391752690000002 podStartE2EDuration="5.468485653s" totalImagesPullingTime="2.929310384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:30.468118177 +0000 UTC m=+56.928609175" watchObservedRunningTime="2026-07-09 15:27:30.468485653 +0000 UTC m=+56.928976650"
Jul 09 15:27:30.492953 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.492926    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493088 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.492989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-serving-certs-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493088 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:30.493253 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493197    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-federate-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493253 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-metrics-client-ca\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493253 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493253 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493530 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw2pc\" (UniqueName: \"kubernetes.io/projected/07adf231-3ea6-485b-8bea-5142b1dfeb98-kube-api-access-sw2pc\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.493530 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.493321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.494140 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.494114    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-serving-certs-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.494372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.494343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.495402 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.495368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/07adf231-3ea6-485b-8bea-5142b1dfeb98-metrics-client-ca\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.496331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.496303    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.496426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.496369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7d058837-d814-40ea-9ef6-82394d96ea51-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-s5dfq\" (UID: \"7d058837-d814-40ea-9ef6-82394d96ea51\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:30.496624 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.496603    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.497445 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.497420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-federate-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.498417 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.498394    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/07adf231-3ea6-485b-8bea-5142b1dfeb98-telemeter-client-tls\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.506121 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.506076    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw2pc\" (UniqueName: \"kubernetes.io/projected/07adf231-3ea6-485b-8bea-5142b1dfeb98-kube-api-access-sw2pc\") pod \"telemeter-client-8485bd4bf-p6sfb\" (UID: \"07adf231-3ea6-485b-8bea-5142b1dfeb98\") " pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.548078 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.548000    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:30.548223 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.548103    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:30.553157 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.553134    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:30.594707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.594676    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"
Jul 09 15:27:30.747157 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:30.747127    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:31.030254 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.030226    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:27:31.079967 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.079935    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:27:31.082685 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.082661    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.096969 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.096941    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:27:31.100149 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100272 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100221    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvzs7\" (UniqueName: \"kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100505    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100533    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.100589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.100561    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.201625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvzs7\" (UniqueName: \"kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.201625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201621    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.201775 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201651    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.201775 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.201775 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201721    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.202179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.201963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.202179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.202026    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.202432 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.202403    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.202789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.202700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.203263 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.203239    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.203512 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.203491    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.204384 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.204365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.204724 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.204704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.228370 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.228347    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvzs7\" (UniqueName: \"kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7\") pod \"console-7f99dc6b56-jzkb5\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") " pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.330237 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.329721    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-thqtm"
Jul 09 15:27:31.396259 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.395938    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:31.402844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.401508    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:27:31.409836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.408985    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.415988 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.414503    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 15:27:31.415988 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.415039    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 15:27:31.415988 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.415274    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 15:27:31.417780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.417333    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 15:27:31.417780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.417562    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 15:27:31.418031 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.417895    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 15:27:31.418144 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.418109    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 15:27:31.418531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.418321    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sjl72\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.419201    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.419412    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.419668    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.419766    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bade8jnvl1pmc\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.420026    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 15:27:31.420842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.420069    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 15:27:31.421320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.421274    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 15:27:31.446647 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.442598    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:27:31.456352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.456144    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 15:27:31.477243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.477164    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"]
Jul 09 15:27:31.492775 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.489219    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8485bd4bf-p6sfb"]
Jul 09 15:27:31.498969 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.498630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"}
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511368    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511477    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511635    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6zsl\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511670    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511807    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.511867 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511953    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.511982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.512035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.512231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.512080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.521643 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.521617    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"bead81b43c0709cce5379f7d6e69b3f13c81e59ea156301023952af569a6396d"}
Jul 09 15:27:31.532313 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.531614    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.612889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613047    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613131    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613184    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613457    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613550    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.614311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613585    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.615225 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613606    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.615225 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613630    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.615225 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.613688    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6zsl\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.615483    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.616112    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.616569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.617309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.617602    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.617630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.618007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.619484    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.620316 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.620179    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.621840 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.621280    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.621840 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.621731    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.623417 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.622513    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.623417 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.622658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.624846 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.623920    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.624846 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.623979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.625473 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.625203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.626303 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.626117    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.627183 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.626976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.627183 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.627151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.629113 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.629073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6zsl\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl\") pod \"prometheus-k8s-0\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.630312 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.630285    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:27:31.778278 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.777956    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:31.962738 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:31.962708    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:27:31.966574 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:31.966538    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7804cfc6_dacd_467e_88f9_c3bee4b07be7.slice/crio-9e5e1d34d550a76e13b873e2bee8125ccb6b7f8fef8eaccbc941fc07de1a45fd WatchSource:0}: Error finding container 9e5e1d34d550a76e13b873e2bee8125ccb6b7f8fef8eaccbc941fc07de1a45fd: Status 404 returned error can't find the container with id 9e5e1d34d550a76e13b873e2bee8125ccb6b7f8fef8eaccbc941fc07de1a45fd
Jul 09 15:27:32.539973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.535601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"}
Jul 09 15:27:32.539973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.535743    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"}
Jul 09 15:27:32.539973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.535825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"}
Jul 09 15:27:32.539973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.535876    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"}
Jul 09 15:27:32.543863 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.543707    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"a10fbe24454670349ee44c799eeaeaac1dc2f5a4e6c2084cf198c2a7e7c2c742"}
Jul 09 15:27:32.543863 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.543801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"34bdac71589b9f36c1624e2dfba8a332a95b765c68af59384235fffda1d71b42"}
Jul 09 15:27:32.545368 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.545305    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq" event={"ID":"7d058837-d814-40ea-9ef6-82394d96ea51","Type":"ContainerStarted","Data":"d62ce456e09b562d25fa8e24af6db632c41634e67753b141507207817cc95493"}
Jul 09 15:27:32.547783 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.547400    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="004d43b5e9515d4f62f2741c6b04be78f459cc174c3c1dcc03a259eac55835e7" exitCode=0
Jul 09 15:27:32.547783 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.547466    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"004d43b5e9515d4f62f2741c6b04be78f459cc174c3c1dcc03a259eac55835e7"}
Jul 09 15:27:32.547783 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.547521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"9e5e1d34d550a76e13b873e2bee8125ccb6b7f8fef8eaccbc941fc07de1a45fd"}
Jul 09 15:27:32.551829 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.551700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f99dc6b56-jzkb5" event={"ID":"3afdadce-4b2d-48d9-aca3-f6e733ea72f2","Type":"ContainerStarted","Data":"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"}
Jul 09 15:27:32.551829 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.551737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f99dc6b56-jzkb5" event={"ID":"3afdadce-4b2d-48d9-aca3-f6e733ea72f2","Type":"ContainerStarted","Data":"05bdf41ef54116231bdf89f196ea13bb53b588e11d215e7d035f9318b97ff5e6"}
Jul 09 15:27:32.553334 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.553310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb" event={"ID":"07adf231-3ea6-485b-8bea-5142b1dfeb98","Type":"ContainerStarted","Data":"bf1f1bfef18ad6884c2dd17873e8cee9a6c731f0036319d588dc64e98433a487"}
Jul 09 15:27:32.635576 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:32.635524    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f99dc6b56-jzkb5" podStartSLOduration=1.63550759 podStartE2EDuration="1.63550759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:32.635030932 +0000 UTC m=+59.095521940" watchObservedRunningTime="2026-07-09 15:27:32.63550759 +0000 UTC m=+59.095998586"
Jul 09 15:27:33.567709 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.567675    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerStarted","Data":"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"}
Jul 09 15:27:33.572717 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.572686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"7e247a10902f4edbda9f7fb1bc9bb0597258fec1a6a0bae43658945db059678a"}
Jul 09 15:27:33.572847 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.572780    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"66cd395682a05a0596b248996dcccac23c50ef2b2d5f6a47d1613069bdeb0cff"}
Jul 09 15:27:33.577044 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.574801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq" event={"ID":"7d058837-d814-40ea-9ef6-82394d96ea51","Type":"ContainerStarted","Data":"1eb95b7d881d4be310799b79821b97348ca558caaee1bf6987460c95beb91c88"}
Jul 09 15:27:33.577044 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.575720    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:33.578413 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.578388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf" event={"ID":"f13c58b3-a475-4cc1-8f9c-65df8b952cda","Type":"ContainerStarted","Data":"954e28bdb69abf317d24085e00348ae56a1eb932da5decf1d6f264fc4c5a1aad"}
Jul 09 15:27:33.582451 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.582408    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq"
Jul 09 15:27:33.605844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.605587    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.83169391 podStartE2EDuration="7.605570402s" totalImagesPullingTime="5.754619365s" totalInitContainerRuntime="19.257127ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:33.60422617 +0000 UTC m=+60.064717171" watchObservedRunningTime="2026-07-09 15:27:33.605570402 +0000 UTC m=+60.066061399"
Jul 09 15:27:33.632915 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.632850    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-s5dfq" podStartSLOduration=2.827357286 podStartE2EDuration="4.632829442s" totalImagesPullingTime="1.805472156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:33.630272696 +0000 UTC m=+60.090763691" watchObservedRunningTime="2026-07-09 15:27:33.632829442 +0000 UTC m=+60.093320443"
Jul 09 15:27:33.657862 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:33.657794    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf" podStartSLOduration=1.322822671 podStartE2EDuration="4.657770454s" totalImagesPullingTime="3.334947783s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:33.656804574 +0000 UTC m=+60.117295573" watchObservedRunningTime="2026-07-09 15:27:33.657770454 +0000 UTC m=+60.118261448"
Jul 09 15:27:34.587158 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.587060    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" event={"ID":"f3abd3de-08f5-42d9-a560-19690bbccd3c","Type":"ContainerStarted","Data":"97311d90ec13648edc75427136f164743faac4e2fcc1673e359ba96ea4069f77"}
Jul 09 15:27:34.587601 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.587247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:34.590110 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.590064    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb" event={"ID":"07adf231-3ea6-485b-8bea-5142b1dfeb98","Type":"ContainerStarted","Data":"e565056448e89a42bb162c4b31b6b483087e5b4384ef2c92dc0de4d7b810dd56"}
Jul 09 15:27:34.590216 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.590122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb" event={"ID":"07adf231-3ea6-485b-8bea-5142b1dfeb98","Type":"ContainerStarted","Data":"c2b291ebee6122bd817fe6315371ab8ec2a5ee1340342dba39517a07e4cc63e5"}
Jul 09 15:27:34.590216 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.590150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb" event={"ID":"07adf231-3ea6-485b-8bea-5142b1dfeb98","Type":"ContainerStarted","Data":"d7f65095a81f5de3aed2ba23f15fa9636f004131b16efc92b3e113d47feb3100"}
Jul 09 15:27:34.617203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.615277    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t" podStartSLOduration=2.244097185 podStartE2EDuration="7.615260002s" totalImagesPullingTime="5.371162817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:34.614338636 +0000 UTC m=+61.074829637" watchObservedRunningTime="2026-07-09 15:27:34.615260002 +0000 UTC m=+61.075750998"
Jul 09 15:27:34.640766 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:34.640702    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8485bd4bf-p6sfb" podStartSLOduration=1.898755261 podStartE2EDuration="4.640687898s" totalImagesPullingTime="2.741932637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:34.640598778 +0000 UTC m=+61.101089777" watchObservedRunningTime="2026-07-09 15:27:34.640687898 +0000 UTC m=+61.101178895"
Jul 09 15:27:35.264648 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.264622    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qprmf"
Jul 09 15:27:35.354073 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.353357    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:27:35.397189 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.397152    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:27:35.413014 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.412987    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.413426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.413400    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:27:35.456463 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456435    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svh9t\" (UniqueName: \"kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456611    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456643    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.456824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.456728    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.559677 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.559582    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.559874 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.559700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.559874 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.559744    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.559993 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.559915    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.559993 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.559955    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svh9t\" (UniqueName: \"kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.560093 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.560000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.560093 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.560028    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.560370 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.560340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.560854 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.560832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.561065 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.561046    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.561207 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.561193    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.564125 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.564033    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.564125 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.564088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.570472 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.570446    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svh9t\" (UniqueName: \"kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t\") pod \"console-5946fd44fb-d5kpm\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") " pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:35.724819 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:35.724785    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:36.749247 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:36.749220    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:27:36.753307 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:36.753276    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8cfff4c_fdd2_419b_915f_2b188317ce19.slice/crio-45f8c1d100afa718127e5802a2fc37134ea4f74668e83ed44036739ea57dca1d WatchSource:0}: Error finding container 45f8c1d100afa718127e5802a2fc37134ea4f74668e83ed44036739ea57dca1d: Status 404 returned error can't find the container with id 45f8c1d100afa718127e5802a2fc37134ea4f74668e83ed44036739ea57dca1d
Jul 09 15:27:37.613486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613448    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"7d00162cfd9167d88c4870d31a4177558f052bc1a17c69eb8a4abf682030c76c"}
Jul 09 15:27:37.613675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"af7eb2a6aab94b66811cb0f5494687d7eec2d3433c655c45409f7ff414415486"}
Jul 09 15:27:37.613675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"e8ba0b788e3e21cbccd044d49689604e9829eaa03f17b2e9b81db726cb811968"}
Jul 09 15:27:37.613675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"ecf3690acd40017e369d513f2819b1b4662ef4d4d274d35b4ffd386a0426e690"}
Jul 09 15:27:37.613675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"77e10320e5e932f8e490d7509e48a1404c9140aa70bee85ff32eb2256a023da4"}
Jul 09 15:27:37.613675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.613648    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerStarted","Data":"92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d"}
Jul 09 15:27:37.615071 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.615049    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5946fd44fb-d5kpm" event={"ID":"d8cfff4c-fdd2-419b-915f-2b188317ce19","Type":"ContainerStarted","Data":"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"}
Jul 09 15:27:37.615169 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.615085    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5946fd44fb-d5kpm" event={"ID":"d8cfff4c-fdd2-419b-915f-2b188317ce19","Type":"ContainerStarted","Data":"45f8c1d100afa718127e5802a2fc37134ea4f74668e83ed44036739ea57dca1d"}
Jul 09 15:27:37.651808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.651732    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.526182479 podStartE2EDuration="6.651711303s" totalImagesPullingTime="4.104623163s" totalInitContainerRuntime="20.905661ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:37.650152062 +0000 UTC m=+64.110643060" watchObservedRunningTime="2026-07-09 15:27:37.651711303 +0000 UTC m=+64.112202302"
Jul 09 15:27:37.672536 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:37.672494    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5946fd44fb-d5kpm" podStartSLOduration=2.672480064 podStartE2EDuration="2.672480064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:27:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:37.671095628 +0000 UTC m=+64.131586625" watchObservedRunningTime="2026-07-09 15:27:37.672480064 +0000 UTC m=+64.132971061"
Jul 09 15:27:39.802076 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.802037    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:39.802506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.802090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:39.804926 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.804910    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 15:27:39.805980 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.805967    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 15:27:39.815493 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.815468    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d446ecc-426d-465e-96b6-d42360901d7f-metrics-certs\") pod \"network-metrics-daemon-9ktpc\" (UID: \"3d446ecc-426d-465e-96b6-d42360901d7f\") " pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:39.815593 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.815474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6660f188-abdc-44da-8487-cadab2eb96e7-original-pull-secret\") pod \"global-pull-secret-syncer-h42wx\" (UID: \"6660f188-abdc-44da-8487-cadab2eb96e7\") " pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:39.903296 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.903266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:39.906293 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.906276    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 15:27:39.916771 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.916734    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 15:27:39.926561 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:39.926542    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frfcr\" (UniqueName: \"kubernetes.io/projected/734914ef-7a03-4867-b5f1-a6cbf0e26d78-kube-api-access-frfcr\") pod \"network-check-target-vg5h6\" (UID: \"734914ef-7a03-4867-b5f1-a6cbf0e26d78\") " pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:40.015290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.015269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5zg4p\""
Jul 09 15:27:40.018686 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.018665    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-krl5b\""
Jul 09 15:27:40.022975 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.022959    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:40.026624 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.026605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9ktpc"
Jul 09 15:27:40.113700 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.113667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h42wx"
Jul 09 15:27:40.170667 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.170647    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vg5h6"]
Jul 09 15:27:40.173023 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:40.172999    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod734914ef_7a03_4867_b5f1_a6cbf0e26d78.slice/crio-d806bc9773f267aaf2d531077058e7fc1bcd6ed5fbba7428b1b1b70ad402140b WatchSource:0}: Error finding container d806bc9773f267aaf2d531077058e7fc1bcd6ed5fbba7428b1b1b70ad402140b: Status 404 returned error can't find the container with id d806bc9773f267aaf2d531077058e7fc1bcd6ed5fbba7428b1b1b70ad402140b
Jul 09 15:27:40.190263 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.190162    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9ktpc"]
Jul 09 15:27:40.192808 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:40.192738    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d446ecc_426d_465e_96b6_d42360901d7f.slice/crio-432c3e5282e444591db86403a0d181a5910f72fdd295ae0bd40fea017b94964e WatchSource:0}: Error finding container 432c3e5282e444591db86403a0d181a5910f72fdd295ae0bd40fea017b94964e: Status 404 returned error can't find the container with id 432c3e5282e444591db86403a0d181a5910f72fdd295ae0bd40fea017b94964e
Jul 09 15:27:40.263641 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.263607    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h42wx"]
Jul 09 15:27:40.266652 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:27:40.266625    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6660f188_abdc_44da_8487_cadab2eb96e7.slice/crio-ce5d1532d7d940ac84262d08d21a6e2051709a81e8ff0ac5bc6d493b8ac593f9 WatchSource:0}: Error finding container ce5d1532d7d940ac84262d08d21a6e2051709a81e8ff0ac5bc6d493b8ac593f9: Status 404 returned error can't find the container with id ce5d1532d7d940ac84262d08d21a6e2051709a81e8ff0ac5bc6d493b8ac593f9
Jul 09 15:27:40.603111 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.603084    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-684ff48bb7-l946t"
Jul 09 15:27:40.630579 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.630543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ktpc" event={"ID":"3d446ecc-426d-465e-96b6-d42360901d7f","Type":"ContainerStarted","Data":"432c3e5282e444591db86403a0d181a5910f72fdd295ae0bd40fea017b94964e"}
Jul 09 15:27:40.633259 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.633230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vg5h6" event={"ID":"734914ef-7a03-4867-b5f1-a6cbf0e26d78","Type":"ContainerStarted","Data":"d806bc9773f267aaf2d531077058e7fc1bcd6ed5fbba7428b1b1b70ad402140b"}
Jul 09 15:27:40.638022 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:40.637991    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h42wx" event={"ID":"6660f188-abdc-44da-8487-cadab2eb96e7","Type":"ContainerStarted","Data":"ce5d1532d7d940ac84262d08d21a6e2051709a81e8ff0ac5bc6d493b8ac593f9"}
Jul 09 15:27:41.397040 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:41.396959    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:27:41.646546 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:41.646509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ktpc" event={"ID":"3d446ecc-426d-465e-96b6-d42360901d7f","Type":"ContainerStarted","Data":"98c3b2b375dad523dff12c9087de126a44d48be4314b841d9875bebda74664ce"}
Jul 09 15:27:41.646727 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:41.646580    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9ktpc" event={"ID":"3d446ecc-426d-465e-96b6-d42360901d7f","Type":"ContainerStarted","Data":"160a5abf01df36392fa8ceefbe34da81b77e9a0c0f2210a162544e386ae63c81"}
Jul 09 15:27:41.674109 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:41.674055    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9ktpc" podStartSLOduration=66.57744632 podStartE2EDuration="1m7.674033421s" totalImagesPullingTime="1.096587101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:41.670492026 +0000 UTC m=+68.130983024" watchObservedRunningTime="2026-07-09 15:27:41.674033421 +0000 UTC m=+68.134524420"
Jul 09 15:27:41.780375 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:41.780345    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:45.667807 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.667746    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h42wx" event={"ID":"6660f188-abdc-44da-8487-cadab2eb96e7","Type":"ContainerStarted","Data":"c7e33bca36cd5093f241d76d3aa1f919628c36d34f39c93480f27c5af1e8d001"}
Jul 09 15:27:45.669326 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.669308    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vg5h6" event={"ID":"734914ef-7a03-4867-b5f1-a6cbf0e26d78","Type":"ContainerStarted","Data":"3584b9735a2d7507e899b026d80089ba1d9c212b07db070707740f3522a6653a"}
Jul 09 15:27:45.669480 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.669457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:27:45.688394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.688347    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h42wx" podStartSLOduration=66.923209679 podStartE2EDuration="1m11.688333749s" totalImagesPullingTime="4.76512407s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:45.686570609 +0000 UTC m=+72.147061609" watchObservedRunningTime="2026-07-09 15:27:45.688333749 +0000 UTC m=+72.148824746"
Jul 09 15:27:45.706150 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.706107    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vg5h6" podStartSLOduration=66.850405987 podStartE2EDuration="1m11.706093273s" totalImagesPullingTime="4.855687286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:26:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:27:45.704657468 +0000 UTC m=+72.165148483" watchObservedRunningTime="2026-07-09 15:27:45.706093273 +0000 UTC m=+72.166584269"
Jul 09 15:27:45.725744 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.725720    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:45.725875 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.725784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:45.730354 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:45.730333    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:46.678205 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:46.678176    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:27:46.748331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:46.748298    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:27:49.790502 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:49.790469    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:49.790912 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:49.790518    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:27:49.912812 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:49.912787    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:49.980504 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:49.980478    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:50.707297 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:50.707267    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:27:57.576205 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.576079    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c6889fdf8-qtprh" podUID="4314040c-c39c-48df-a337-a24de545376c" containerName="console" containerID="cri-o://011adf8cb92e0e1c300f2252593790aa7d3a68d366c4a8ba6a355fa55a5a4a90" gracePeriod=15
Jul 09 15:27:57.727356 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.727324    2578 generic.go:356] "Generic (PLEG): container finished" podID="4314040c-c39c-48df-a337-a24de545376c" containerID="011adf8cb92e0e1c300f2252593790aa7d3a68d366c4a8ba6a355fa55a5a4a90" exitCode=0
Jul 09 15:27:57.727510 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.727398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6889fdf8-qtprh" event={"ID":"4314040c-c39c-48df-a337-a24de545376c","Type":"ContainerDied","Data":"011adf8cb92e0e1c300f2252593790aa7d3a68d366c4a8ba6a355fa55a5a4a90"}
Jul 09 15:27:57.823334 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.823304    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:57.980078 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cw75p\" (UniqueName: \"kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980078 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980054    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980078 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980072    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980225    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980361    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config" (OuterVolumeSpecName: "console-config") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:27:57.980547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980488    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca" (OuterVolumeSpecName: "service-ca") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:27:57.980547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980414    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert\") pod \"4314040c-c39c-48df-a337-a24de545376c\" (UID: \"4314040c-c39c-48df-a337-a24de545376c\") "
Jul 09 15:27:57.980648 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980544    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:27:57.980798 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980778    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-service-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:57.980798 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980801    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-console-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:57.980993 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980810    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-oauth-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:57.981046 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.980981    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:27:57.982543 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.982523    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p" (OuterVolumeSpecName: "kube-api-access-cw75p") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "kube-api-access-cw75p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:27:57.982652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.982584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:27:57.982729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:57.982713    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4314040c-c39c-48df-a337-a24de545376c" (UID: "4314040c-c39c-48df-a337-a24de545376c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:27:58.081535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.081496    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-oauth-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:58.081535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.081531    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4314040c-c39c-48df-a337-a24de545376c-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:58.081535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.081540    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4314040c-c39c-48df-a337-a24de545376c-console-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:58.081535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.081549    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cw75p\" (UniqueName: \"kubernetes.io/projected/4314040c-c39c-48df-a337-a24de545376c-kube-api-access-cw75p\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:27:58.733290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.733258    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c6889fdf8-qtprh"
Jul 09 15:27:58.733290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.733275    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c6889fdf8-qtprh" event={"ID":"4314040c-c39c-48df-a337-a24de545376c","Type":"ContainerDied","Data":"7527ce5bdafd0e0d8da1760ac53ecc99e187274747f4335c6b417b20f505b5b3"}
Jul 09 15:27:58.733788 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.733329    2578 scope.go:117] "RemoveContainer" containerID="011adf8cb92e0e1c300f2252593790aa7d3a68d366c4a8ba6a355fa55a5a4a90"
Jul 09 15:27:58.754015 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.753990    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:27:58.758675 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:27:58.758642    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c6889fdf8-qtprh"]
Jul 09 15:28:00.104325 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.104294    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4314040c-c39c-48df-a337-a24de545376c" path="/var/lib/kubelet/pods/4314040c-c39c-48df-a337-a24de545376c/volumes"
Jul 09 15:28:00.377888 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.377786    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f99dc6b56-jzkb5" podUID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" containerName="console" containerID="cri-o://ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5" gracePeriod=15
Jul 09 15:28:00.626114 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.626092    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:28:00.704045 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.703975    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704045 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704045 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704034    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704063    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704115    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvzs7\" (UniqueName: \"kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704240    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert\") pod \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\" (UID: \"3afdadce-4b2d-48d9-aca3-f6e733ea72f2\") "
Jul 09 15:28:00.704500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704387    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config" (OuterVolumeSpecName: "console-config") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:00.704500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704452    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:00.704500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704466    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:00.704633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704567    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.704633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704583    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.704633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704592    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-oauth-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.705100 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.704902    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca" (OuterVolumeSpecName: "service-ca") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:00.706439 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.706403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7" (OuterVolumeSpecName: "kube-api-access-fvzs7") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "kube-api-access-fvzs7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:00.706537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.706518    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:00.706614 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.706592    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3afdadce-4b2d-48d9-aca3-f6e733ea72f2" (UID: "3afdadce-4b2d-48d9-aca3-f6e733ea72f2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:00.743209 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.743179    2578 generic.go:356] "Generic (PLEG): container finished" podID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" containerID="ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5" exitCode=0
Jul 09 15:28:00.743301 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.743230    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f99dc6b56-jzkb5"
Jul 09 15:28:00.743301 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.743261    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f99dc6b56-jzkb5" event={"ID":"3afdadce-4b2d-48d9-aca3-f6e733ea72f2","Type":"ContainerDied","Data":"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"}
Jul 09 15:28:00.743368 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.743314    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f99dc6b56-jzkb5" event={"ID":"3afdadce-4b2d-48d9-aca3-f6e733ea72f2","Type":"ContainerDied","Data":"05bdf41ef54116231bdf89f196ea13bb53b588e11d215e7d035f9318b97ff5e6"}
Jul 09 15:28:00.743368 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.743336    2578 scope.go:117] "RemoveContainer" containerID="ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"
Jul 09 15:28:00.752550 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.752534    2578 scope.go:117] "RemoveContainer" containerID="ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"
Jul 09 15:28:00.752861 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:00.752841    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5\": container with ID starting with ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5 not found: ID does not exist" containerID="ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"
Jul 09 15:28:00.752921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.752871    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5"} err="failed to get container status \"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5\": rpc error: code = NotFound desc = could not find container \"ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5\": container with ID starting with ba4e006509a11f0949c33ceeb9eee1510ca0ba4a8e42575f1b922faba7d517d5 not found: ID does not exist"
Jul 09 15:28:00.767011 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.766991    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:28:00.770878 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.770859    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f99dc6b56-jzkb5"]
Jul 09 15:28:00.805693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.805675    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-service-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.805693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.805694    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.805832 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.805704    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-console-oauth-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:00.805832 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:00.805714    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvzs7\" (UniqueName: \"kubernetes.io/projected/3afdadce-4b2d-48d9-aca3-f6e733ea72f2-kube-api-access-fvzs7\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:02.104056 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:02.104019    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" path="/var/lib/kubelet/pods/3afdadce-4b2d-48d9-aca3-f6e733ea72f2/volumes"
Jul 09 15:28:09.796088 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:09.796054    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:28:09.799797 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:09.799775    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6455bc7799-wh8xf"
Jul 09 15:28:11.772910 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:11.772848    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-688cfd7ff7-qdnqs" podUID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" containerName="console" containerID="cri-o://152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4" gracePeriod=15
Jul 09 15:28:12.043904 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.043837    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:28:12.087799 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087743    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9894\" (UniqueName: \"kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.087938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087852    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.087938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087876    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.087938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087896    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.087938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087918    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.088103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.087969    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert\") pod \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\" (UID: \"af1f9fea-bcdc-44aa-898f-d4e20ba468dd\") "
Jul 09 15:28:12.088352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.088323    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:12.088352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.088337    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config" (OuterVolumeSpecName: "console-config") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:12.088596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.088568    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:12.090054 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.090030    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894" (OuterVolumeSpecName: "kube-api-access-l9894") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "kube-api-access-l9894". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:12.090599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.090566    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:12.090690 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.090645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "af1f9fea-bcdc-44aa-898f-d4e20ba468dd" (UID: "af1f9fea-bcdc-44aa-898f-d4e20ba468dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:12.188921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188892    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-oauth-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.188921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188919    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9894\" (UniqueName: \"kubernetes.io/projected/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-kube-api-access-l9894\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.189067 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188933    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-service-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.189067 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188947    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.189067 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188956    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-oauth-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.189067 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.188965    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/af1f9fea-bcdc-44aa-898f-d4e20ba468dd-console-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:12.794433 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.794399    2578 generic.go:356] "Generic (PLEG): container finished" podID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" containerID="152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4" exitCode=0
Jul 09 15:28:12.794842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.794445    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688cfd7ff7-qdnqs"
Jul 09 15:28:12.794842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.794491    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688cfd7ff7-qdnqs" event={"ID":"af1f9fea-bcdc-44aa-898f-d4e20ba468dd","Type":"ContainerDied","Data":"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"}
Jul 09 15:28:12.794842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.794546    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688cfd7ff7-qdnqs" event={"ID":"af1f9fea-bcdc-44aa-898f-d4e20ba468dd","Type":"ContainerDied","Data":"a2f800d5c654c0b7a55f8778eabde42e88adf2d5bfa52b81faa50e32ef0999a1"}
Jul 09 15:28:12.794842 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.794570    2578 scope.go:117] "RemoveContainer" containerID="152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"
Jul 09 15:28:12.804894 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.804853    2578 scope.go:117] "RemoveContainer" containerID="152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"
Jul 09 15:28:12.805326 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:12.805304    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4\": container with ID starting with 152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4 not found: ID does not exist" containerID="152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"
Jul 09 15:28:12.805385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.805340    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4"} err="failed to get container status \"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4\": rpc error: code = NotFound desc = could not find container \"152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4\": container with ID starting with 152301d96685b0928ae1acade0d3e239c8e59e38a96aea2ba5dc1f41779b9cc4 not found: ID does not exist"
Jul 09 15:28:12.816161 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.816140    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:28:12.823158 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:12.823137    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-688cfd7ff7-qdnqs"]
Jul 09 15:28:14.109322 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.109278    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" path="/var/lib/kubelet/pods/af1f9fea-bcdc-44aa-898f-d4e20ba468dd/volumes"
Jul 09 15:28:14.302195 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302166    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:28:14.302546 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302532    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" containerName="console"
Jul 09 15:28:14.302589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302548    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" containerName="console"
Jul 09 15:28:14.302589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302560    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" containerName="console"
Jul 09 15:28:14.302589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302565    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" containerName="console"
Jul 09 15:28:14.302589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302579    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4314040c-c39c-48df-a337-a24de545376c" containerName="console"
Jul 09 15:28:14.302589 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302585    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4314040c-c39c-48df-a337-a24de545376c" containerName="console"
Jul 09 15:28:14.302729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302647    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4314040c-c39c-48df-a337-a24de545376c" containerName="console"
Jul 09 15:28:14.302729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302654    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3afdadce-4b2d-48d9-aca3-f6e733ea72f2" containerName="console"
Jul 09 15:28:14.302729 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.302661    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="af1f9fea-bcdc-44aa-898f-d4e20ba468dd" containerName="console"
Jul 09 15:28:14.305632 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.305614    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.321376 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.321354    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:28:14.404229 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404160    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404229 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404197    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404229 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404226    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404412 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404260    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404412 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dhgr\" (UniqueName: \"kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404412 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404402    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.404502 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.404426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505609 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505570    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dhgr\" (UniqueName: \"kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505653    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.505998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.505809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.506601 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.506575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.506705 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.506575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.506841 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.506824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.506884 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.506833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.508323 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.508301    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.508426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.508408    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.516211 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.516190    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dhgr\" (UniqueName: \"kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr\") pod \"console-5d4ffcfbd8-z29kw\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") " pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.615328 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.615298    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:14.745952 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.745929    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:28:14.748252 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:28:14.748225    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4123ea31_d43c_4c08_910c_5ed7c96e45ed.slice/crio-6863feb61c2f3e7c1293a25573c7696d5010cbd1784f79437af6cf39fc0e45a4 WatchSource:0}: Error finding container 6863feb61c2f3e7c1293a25573c7696d5010cbd1784f79437af6cf39fc0e45a4: Status 404 returned error can't find the container with id 6863feb61c2f3e7c1293a25573c7696d5010cbd1784f79437af6cf39fc0e45a4
Jul 09 15:28:14.805456 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.805428    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4ffcfbd8-z29kw" event={"ID":"4123ea31-d43c-4c08-910c-5ed7c96e45ed","Type":"ContainerStarted","Data":"6863feb61c2f3e7c1293a25573c7696d5010cbd1784f79437af6cf39fc0e45a4"}
Jul 09 15:28:14.830011 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.829965    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d4ffcfbd8-z29kw" podStartSLOduration=0.829945327 podStartE2EDuration="829.945327ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:28:14.829122083 +0000 UTC m=+101.289613080" watchObservedRunningTime="2026-07-09 15:28:14.829945327 +0000 UTC m=+101.290436324"
Jul 09 15:28:14.930884 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.930805    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:14.931127 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931086    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="alertmanager" containerID="cri-o://c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b" gracePeriod=120
Jul 09 15:28:14.931190 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931157    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-metric" containerID="cri-o://c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb" gracePeriod=120
Jul 09 15:28:14.931239 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931172    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="config-reloader" containerID="cri-o://36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9" gracePeriod=120
Jul 09 15:28:14.931239 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931208    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy" containerID="cri-o://5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1" gracePeriod=120
Jul 09 15:28:14.931329 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931172    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-web" containerID="cri-o://c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311" gracePeriod=120
Jul 09 15:28:14.931329 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:14.931262    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="prom-label-proxy" containerID="cri-o://5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1" gracePeriod=120
Jul 09 15:28:15.812465 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.812383    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4ffcfbd8-z29kw" event={"ID":"4123ea31-d43c-4c08-910c-5ed7c96e45ed","Type":"ContainerStarted","Data":"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"}
Jul 09 15:28:15.815670 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815646    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1" exitCode=0
Jul 09 15:28:15.815670 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815666    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb" exitCode=0
Jul 09 15:28:15.815670 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815673    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1" exitCode=0
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815679    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9" exitCode=0
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815684    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b" exitCode=0
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815712    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"}
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"}
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815802    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"}
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815826    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"}
Jul 09 15:28:15.815857 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:15.815849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"}
Jul 09 15:28:16.176847 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.176822    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:16.221159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221131    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221322 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221191    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221322 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221227    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221322 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221275    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221322 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221314    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221351    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221384    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrpvr\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221418    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221462    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221486    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221535    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221861 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221582    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.221861 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.221613    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config\") pod \"e027b13f-1d3f-40a5-a38e-6269063ea80a\" (UID: \"e027b13f-1d3f-40a5-a38e-6269063ea80a\") "
Jul 09 15:28:16.222276 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.222035    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:16.222854 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.222623    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:28:16.223528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.223275    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:16.226586 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.226559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out" (OuterVolumeSpecName: "config-out") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:28:16.226915 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.226881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume" (OuterVolumeSpecName: "config-volume") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.227156 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227049    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.227417 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227390    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.227559 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227542    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.227657 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227552    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr" (OuterVolumeSpecName: "kube-api-access-vrpvr") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "kube-api-access-vrpvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:16.227721 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.227808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.227721    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:16.234225 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.234198    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.240979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.240959    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config" (OuterVolumeSpecName: "web-config") pod "e027b13f-1d3f-40a5-a38e-6269063ea80a" (UID: "e027b13f-1d3f-40a5-a38e-6269063ea80a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:16.322601 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322570    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-tls-assets\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322601 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322594    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-out\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322601 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322603    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322614    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrpvr\" (UniqueName: \"kubernetes.io/projected/e027b13f-1d3f-40a5-a38e-6269063ea80a-kube-api-access-vrpvr\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322625    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-main-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322635    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-metrics-client-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322644    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-cluster-tls-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322652    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322661    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-main-db\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322671    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-web-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322678    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-config-volume\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322686    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e027b13f-1d3f-40a5-a38e-6269063ea80a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.322844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.322695    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e027b13f-1d3f-40a5-a38e-6269063ea80a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:16.677346 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.677319    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vg5h6"
Jul 09 15:28:16.823816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.823780    2578 generic.go:356] "Generic (PLEG): container finished" podID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerID="c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311" exitCode=0
Jul 09 15:28:16.824159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.823878    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:16.824159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.823873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"}
Jul 09 15:28:16.824159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.823946    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e027b13f-1d3f-40a5-a38e-6269063ea80a","Type":"ContainerDied","Data":"010855fc888fadfe5edeb888eede36ce11de34ec09dfe6b65e776459361e3358"}
Jul 09 15:28:16.824159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.824020    2578 scope.go:117] "RemoveContainer" containerID="5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"
Jul 09 15:28:16.831968 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.831948    2578 scope.go:117] "RemoveContainer" containerID="c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"
Jul 09 15:28:16.839449 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.839433    2578 scope.go:117] "RemoveContainer" containerID="5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"
Jul 09 15:28:16.847972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.847937    2578 scope.go:117] "RemoveContainer" containerID="c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"
Jul 09 15:28:16.855284 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.855266    2578 scope.go:117] "RemoveContainer" containerID="36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"
Jul 09 15:28:16.862987 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.862967    2578 scope.go:117] "RemoveContainer" containerID="c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"
Jul 09 15:28:16.867792 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.867770    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:16.871617 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.871600    2578 scope.go:117] "RemoveContainer" containerID="eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5"
Jul 09 15:28:16.874746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.874728    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:16.880992 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.880977    2578 scope.go:117] "RemoveContainer" containerID="5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"
Jul 09 15:28:16.881252 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.881229    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1\": container with ID starting with 5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1 not found: ID does not exist" containerID="5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"
Jul 09 15:28:16.881305 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881269    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1"} err="failed to get container status \"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1\": rpc error: code = NotFound desc = could not find container \"5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1\": container with ID starting with 5c5588245600176b8d326e0e07a0ae79edd42990d594d0b2d448551bde6ac5e1 not found: ID does not exist"
Jul 09 15:28:16.881305 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881287    2578 scope.go:117] "RemoveContainer" containerID="c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"
Jul 09 15:28:16.881538 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.881520    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb\": container with ID starting with c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb not found: ID does not exist" containerID="c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"
Jul 09 15:28:16.881575 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881546    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb"} err="failed to get container status \"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb\": rpc error: code = NotFound desc = could not find container \"c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb\": container with ID starting with c7f766262e86d894056426ef9968a1264593ca08b789e9f1a91ae1068d3022bb not found: ID does not exist"
Jul 09 15:28:16.881575 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881564    2578 scope.go:117] "RemoveContainer" containerID="5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"
Jul 09 15:28:16.881842 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.881823    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1\": container with ID starting with 5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1 not found: ID does not exist" containerID="5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"
Jul 09 15:28:16.881939 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881845    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1"} err="failed to get container status \"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1\": rpc error: code = NotFound desc = could not find container \"5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1\": container with ID starting with 5e28e6acd61a5338c5ecb978a52621656ef3efdea73a9587bb9b894ac9c53db1 not found: ID does not exist"
Jul 09 15:28:16.881939 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.881860    2578 scope.go:117] "RemoveContainer" containerID="c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"
Jul 09 15:28:16.882137 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.882119    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311\": container with ID starting with c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311 not found: ID does not exist" containerID="c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"
Jul 09 15:28:16.882194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882141    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311"} err="failed to get container status \"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311\": rpc error: code = NotFound desc = could not find container \"c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311\": container with ID starting with c6cc9f6497e7df54fb86df5b5d598cce614c5f7fede51e3ec002e4f9c3ab7311 not found: ID does not exist"
Jul 09 15:28:16.882194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882154    2578 scope.go:117] "RemoveContainer" containerID="36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"
Jul 09 15:28:16.882401 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.882386    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9\": container with ID starting with 36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9 not found: ID does not exist" containerID="36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"
Jul 09 15:28:16.882466 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882406    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9"} err="failed to get container status \"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9\": rpc error: code = NotFound desc = could not find container \"36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9\": container with ID starting with 36e047f9e5ffe9152dcc6dd9105168c1eed644519686cf60778e3204172cebb9 not found: ID does not exist"
Jul 09 15:28:16.882466 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882419    2578 scope.go:117] "RemoveContainer" containerID="c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"
Jul 09 15:28:16.882647 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.882629    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b\": container with ID starting with c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b not found: ID does not exist" containerID="c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"
Jul 09 15:28:16.882710 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882655    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b"} err="failed to get container status \"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b\": rpc error: code = NotFound desc = could not find container \"c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b\": container with ID starting with c078d98e67b089da86633a43e19d0984d4d66c8b989e52828dff61107837cd7b not found: ID does not exist"
Jul 09 15:28:16.882710 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882676    2578 scope.go:117] "RemoveContainer" containerID="eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5"
Jul 09 15:28:16.882962 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:16.882945    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5\": container with ID starting with eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5 not found: ID does not exist" containerID="eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5"
Jul 09 15:28:16.883015 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.882967    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5"} err="failed to get container status \"eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5\": rpc error: code = NotFound desc = could not find container \"eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5\": container with ID starting with eca0be71e69824d231024059364bfa8ce38047672392f53728b509e450cc24f5 not found: ID does not exist"
Jul 09 15:28:16.917928 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.917898    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:16.918290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918274    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="alertmanager"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918293    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="alertmanager"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918303    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-metric"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918311    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-metric"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918327    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="config-reloader"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918335    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="config-reloader"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918346    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918354    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918362    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="prom-label-proxy"
Jul 09 15:28:16.918372 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918371    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="prom-label-proxy"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918396    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-web"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918405    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-web"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918419    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="init-config-reloader"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918427    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="init-config-reloader"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918527    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="config-reloader"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918541    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="prom-label-proxy"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918555    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-metric"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918565    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy-web"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918573    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="alertmanager"
Jul 09 15:28:16.918808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.918581    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" containerName="kube-rbac-proxy"
Jul 09 15:28:16.924651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.924628    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:16.927779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.927739    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 09 15:28:16.928217 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928025    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 09 15:28:16.928217 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928047    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 09 15:28:16.928217 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928082    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 09 15:28:16.928217 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928126    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7chhh\""
Jul 09 15:28:16.928217 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928136    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 09 15:28:16.928524 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928423    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 09 15:28:16.928574 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928524    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 09 15:28:16.928813 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.928791    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 09 15:28:16.936255 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.936237    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 09 15:28:16.947741 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:16.947667    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:17.029828 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.029798    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-out\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.029828 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.029831    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.029849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.029873    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.029974    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-volume\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030040    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b7q4\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-kube-api-access-4b7q4\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030178    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030241    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030277    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-web-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030305    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.030459 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.030373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.131150 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-out\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.131288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.131288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.131288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.131288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-volume\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131330    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b7q4\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-kube-api-access-4b7q4\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131379    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131493    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-web-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131520    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.132228 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.131995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.136086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.132276    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.136086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.132440    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1006d858-b8d6-40f3-b8b3-adfde31d6e62-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.136086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.135133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.136086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.135659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-web-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.136290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.136143    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-volume\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.142737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.136694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.142737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.137100    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1006d858-b8d6-40f3-b8b3-adfde31d6e62-config-out\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.142737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.137170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.142737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.137285    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.143392 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.143108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.143392 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.143341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b7q4\" (UniqueName: \"kubernetes.io/projected/1006d858-b8d6-40f3-b8b3-adfde31d6e62-kube-api-access-4b7q4\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.145086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.145060    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1006d858-b8d6-40f3-b8b3-adfde31d6e62-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1006d858-b8d6-40f3-b8b3-adfde31d6e62\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.235925 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.235891    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 09 15:28:17.399018 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.398987    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 09 15:28:17.400996 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:28:17.400969    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1006d858_b8d6_40f3_b8b3_adfde31d6e62.slice/crio-fc9a7dab6d0c4c606880cd5c873b65e553df0f44b175b578a57c254c765f7a86 WatchSource:0}: Error finding container fc9a7dab6d0c4c606880cd5c873b65e553df0f44b175b578a57c254c765f7a86: Status 404 returned error can't find the container with id fc9a7dab6d0c4c606880cd5c873b65e553df0f44b175b578a57c254c765f7a86
Jul 09 15:28:17.830193 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.830160    2578 generic.go:356] "Generic (PLEG): container finished" podID="1006d858-b8d6-40f3-b8b3-adfde31d6e62" containerID="b019b60fb781d0eeae833659e2fef273f1e73882623eb76ceb9a38ec9913715e" exitCode=0
Jul 09 15:28:17.830644 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.830249    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerDied","Data":"b019b60fb781d0eeae833659e2fef273f1e73882623eb76ceb9a38ec9913715e"}
Jul 09 15:28:17.830644 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:17.830339    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"fc9a7dab6d0c4c606880cd5c873b65e553df0f44b175b578a57c254c765f7a86"}
Jul 09 15:28:18.104710 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.104679    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e027b13f-1d3f-40a5-a38e-6269063ea80a" path="/var/lib/kubelet/pods/e027b13f-1d3f-40a5-a38e-6269063ea80a/volumes"
Jul 09 15:28:18.844126 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844082    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"16bef5cdd7698da54ad7b61826f6dc8f96c2433b87bee24481ec8d8439c767b7"}
Jul 09 15:28:18.844587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844181    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"e280289c333a13f7d380835c17b29271bef73e2dde870277273ddfb43563535f"}
Jul 09 15:28:18.844587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844227    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"e517573e8c7f683218eab432a8692f68e48e3cda0e16db1f1066e88f28e05b7d"}
Jul 09 15:28:18.844587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844267    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"edd8f7023ef6a6e8a569b283593c12c8fc798bfa516d1f2686910cf3fd5623ce"}
Jul 09 15:28:18.844587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"0efc0ee294164bae1241a45be8a046d3be025a461dfaf814e4e199d82c3bc145"}
Jul 09 15:28:18.844587 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.844354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1006d858-b8d6-40f3-b8b3-adfde31d6e62","Type":"ContainerStarted","Data":"374cbeb277a5c14915e5165f6fed3ea27ca6821fced700738f7bf1803c2d1a50"}
Jul 09 15:28:18.877831 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:18.877770    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.860186962 podStartE2EDuration="2.877729744s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.542782ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:28:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:28:18.87611969 +0000 UTC m=+105.336610724" watchObservedRunningTime="2026-07-09 15:28:18.877729744 +0000 UTC m=+105.338220742"
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.267802    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268377    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="prometheus" containerID="cri-o://92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d" gracePeriod=600
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268634    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy" containerID="cri-o://af7eb2a6aab94b66811cb0f5494687d7eec2d3433c655c45409f7ff414415486" gracePeriod=600
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268838    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-web" containerID="cri-o://e8ba0b788e3e21cbccd044d49689604e9829eaa03f17b2e9b81db726cb811968" gracePeriod=600
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268869    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-thanos" containerID="cri-o://7d00162cfd9167d88c4870d31a4177558f052bc1a17c69eb8a4abf682030c76c" gracePeriod=600
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268913    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="config-reloader" containerID="cri-o://77e10320e5e932f8e490d7509e48a1404c9140aa70bee85ff32eb2256a023da4" gracePeriod=600
Jul 09 15:28:19.272019 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.268988    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="thanos-sidecar" containerID="cri-o://ecf3690acd40017e369d513f2819b1b4662ef4d4d274d35b4ffd386a0426e690" gracePeriod=600
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856181    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="7d00162cfd9167d88c4870d31a4177558f052bc1a17c69eb8a4abf682030c76c" exitCode=0
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856208    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="af7eb2a6aab94b66811cb0f5494687d7eec2d3433c655c45409f7ff414415486" exitCode=0
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856214    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="ecf3690acd40017e369d513f2819b1b4662ef4d4d274d35b4ffd386a0426e690" exitCode=0
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856220    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="77e10320e5e932f8e490d7509e48a1404c9140aa70bee85ff32eb2256a023da4" exitCode=0
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856228    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d" exitCode=0
Jul 09 15:28:19.856268 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856250    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"7d00162cfd9167d88c4870d31a4177558f052bc1a17c69eb8a4abf682030c76c"}
Jul 09 15:28:19.856998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"af7eb2a6aab94b66811cb0f5494687d7eec2d3433c655c45409f7ff414415486"}
Jul 09 15:28:19.856998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"ecf3690acd40017e369d513f2819b1b4662ef4d4d274d35b4ffd386a0426e690"}
Jul 09 15:28:19.856998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"77e10320e5e932f8e490d7509e48a1404c9140aa70bee85ff32eb2256a023da4"}
Jul 09 15:28:19.856998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:19.856436    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d"}
Jul 09 15:28:20.692884 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:20.692800    2578 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d is running failed: container process not found" containerID="92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 09 15:28:20.693205 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:20.693173    2578 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d is running failed: container process not found" containerID="92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 09 15:28:20.693522 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:20.693499    2578 log.go:32] "ExecSync cmd from runtime service failed" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d is running failed: container process not found" containerID="92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d" cmd=["sh","-c","if [ -x \"$(command -v curl)\" ]; then exec curl --fail http://localhost:9090/-/ready; elif [ -x \"$(command -v wget)\" ]; then exec wget -q -O /dev/null http://localhost:9090/-/ready; else exit 1; fi"]
Jul 09 15:28:20.693619 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:20.693537    2578 prober.go:104] "Probe errored" err="rpc error: code = NotFound desc = container is not created or running: checking if PID of 92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d is running failed: container process not found" probeType="Readiness" pod="openshift-monitoring/prometheus-k8s-0" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="prometheus" probeResult="unknown"
Jul 09 15:28:20.871509 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:20.871479    2578 generic.go:356] "Generic (PLEG): container finished" podID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerID="e8ba0b788e3e21cbccd044d49689604e9829eaa03f17b2e9b81db726cb811968" exitCode=0
Jul 09 15:28:20.871855 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:20.871549    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"e8ba0b788e3e21cbccd044d49689604e9829eaa03f17b2e9b81db726cb811968"}
Jul 09 15:28:21.085737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.085712    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176015    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176101    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176135    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176172    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176198    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176255    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176284    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176314    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176362    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176423    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176459    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176490    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176546    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6zsl\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176576    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.177302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176617    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.178380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176664    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.178380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.176694    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config\") pod \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\" (UID: \"7804cfc6-dacd-467e-88f9-c3bee4b07be7\") "
Jul 09 15:28:21.178380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.177559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.180799 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.180042    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:28:21.180799 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.180506    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.182627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.182600    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.183901 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.183065    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.183901 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.183608    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.186586 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.185772    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.186586 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.186308    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:21.200005 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.199976    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.200911 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.200870    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.201014 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.200990    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl" (OuterVolumeSpecName: "kube-api-access-z6zsl") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "kube-api-access-z6zsl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:21.201202 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201156    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config" (OuterVolumeSpecName: "config") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.201279 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201264    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out" (OuterVolumeSpecName: "config-out") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:28:21.201516 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201495    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:21.201625 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201522    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.201707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201634    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.201927 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201896    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.202025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.201947    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.202097 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.202083    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.211205 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.211177    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config" (OuterVolumeSpecName: "web-config") pod "7804cfc6-dacd-467e-88f9-c3bee4b07be7" (UID: "7804cfc6-dacd-467e-88f9-c3bee4b07be7"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:21.277403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277370    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277404    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6zsl\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-kube-api-access-z6zsl\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277421    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277436    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277453    2578 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277465    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-web-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277478    2578 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-kube-rbac-proxy\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277492    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-db\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277505    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277518    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7804cfc6-dacd-467e-88f9-c3bee4b07be7-config-out\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277531    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277544    2578 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-metrics-client-certs\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277557    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-metrics-client-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277569    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7804cfc6-dacd-467e-88f9-c3bee4b07be7-tls-assets\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277582    2578 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277596    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277611    2578 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277628    2578 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.277642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277645    2578 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/7804cfc6-dacd-467e-88f9-c3bee4b07be7-secret-grpc-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.278240 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.277661    2578 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7804cfc6-dacd-467e-88f9-c3bee4b07be7-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:21.879692 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.879670    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:21.880096 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.879666    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"7804cfc6-dacd-467e-88f9-c3bee4b07be7","Type":"ContainerDied","Data":"9e5e1d34d550a76e13b873e2bee8125ccb6b7f8fef8eaccbc941fc07de1a45fd"}
Jul 09 15:28:21.880096 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.879822    2578 scope.go:117] "RemoveContainer" containerID="7d00162cfd9167d88c4870d31a4177558f052bc1a17c69eb8a4abf682030c76c"
Jul 09 15:28:21.888719 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.888702    2578 scope.go:117] "RemoveContainer" containerID="af7eb2a6aab94b66811cb0f5494687d7eec2d3433c655c45409f7ff414415486"
Jul 09 15:28:21.900049 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.900030    2578 scope.go:117] "RemoveContainer" containerID="e8ba0b788e3e21cbccd044d49689604e9829eaa03f17b2e9b81db726cb811968"
Jul 09 15:28:21.908105 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.908079    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:21.908555 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.908539    2578 scope.go:117] "RemoveContainer" containerID="ecf3690acd40017e369d513f2819b1b4662ef4d4d274d35b4ffd386a0426e690"
Jul 09 15:28:21.915698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.915675    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:21.916650 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.916527    2578 scope.go:117] "RemoveContainer" containerID="77e10320e5e932f8e490d7509e48a1404c9140aa70bee85ff32eb2256a023da4"
Jul 09 15:28:21.923694 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.923678    2578 scope.go:117] "RemoveContainer" containerID="92e7116ebcf289607ecaca0402a30ad0b4163d6c23c09c624d12d1a0512c4f7d"
Jul 09 15:28:21.931247 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.931232    2578 scope.go:117] "RemoveContainer" containerID="004d43b5e9515d4f62f2741c6b04be78f459cc174c3c1dcc03a259eac55835e7"
Jul 09 15:28:21.951741 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.951718    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:21.952120 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952102    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="config-reloader"
Jul 09 15:28:21.952188 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952123    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="config-reloader"
Jul 09 15:28:21.952188 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952140    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="thanos-sidecar"
Jul 09 15:28:21.952188 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952148    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="thanos-sidecar"
Jul 09 15:28:21.952188 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952164    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy"
Jul 09 15:28:21.952188 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952172    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952190    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-web"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952198    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-web"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952207    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-thanos"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952215    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-thanos"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952237    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="prometheus"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952246    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="prometheus"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952260    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="init-config-reloader"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952270    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="init-config-reloader"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952363    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="config-reloader"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952375    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="prometheus"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952386    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952397    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-thanos"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952406    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="kube-rbac-proxy-web"
Jul 09 15:28:21.952425 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.952417    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" containerName="thanos-sidecar"
Jul 09 15:28:21.955974 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.955957    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:21.959484 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 15:28:21.959564 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959469    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-sjl72\""
Jul 09 15:28:21.959622 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959472    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-bade8jnvl1pmc\""
Jul 09 15:28:21.959791 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959775    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 15:28:21.959883 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959866    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 15:28:21.959938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959879    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 15:28:21.959938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959895    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 15:28:21.959938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.959916    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 15:28:21.961835 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.961819    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 15:28:21.962658 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.962611    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 15:28:21.962658 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.962620    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 15:28:21.962820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.962660    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 15:28:21.962820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.962660    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 15:28:21.963131 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.963114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 15:28:21.968116 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.968100    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 15:28:21.969673 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.969656    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 15:28:21.978269 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:21.978250    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:22.085584 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085622    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.085789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085879    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085939    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5cw5\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-kube-api-access-x5cw5\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.085987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086007 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086024    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086094    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.086210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.086111    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.104777 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.104724    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7804cfc6-dacd-467e-88f9-c3bee4b07be7" path="/var/lib/kubelet/pods/7804cfc6-dacd-467e-88f9-c3bee4b07be7/volumes"
Jul 09 15:28:22.186611 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186611 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186612    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186629    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186646    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186779 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186777    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5cw5\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-kube-api-access-x5cw5\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186822    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186885    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.186962 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186940    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.186987    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187023    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187051    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187117    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.187698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.187577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.189032 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.189004    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.189568 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.189539    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.189932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.189910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config-out\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.190705 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.190619    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192037 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.191141    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-web-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192037 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.191253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192037 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.191465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192037 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.191713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192037 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.191987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192314 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.192131    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-config\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192314 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.192137    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192604 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.192582    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192604 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.192595    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.192950 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.192932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9a8e7f05-26f5-4e04-9252-7133db41dbbf-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.193164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.193149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9a8e7f05-26f5-4e04-9252-7133db41dbbf-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.196503 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.196481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5cw5\" (UniqueName: \"kubernetes.io/projected/9a8e7f05-26f5-4e04-9252-7133db41dbbf-kube-api-access-x5cw5\") pod \"prometheus-k8s-0\" (UID: \"9a8e7f05-26f5-4e04-9252-7133db41dbbf\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.266525 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.266497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:22.410606 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.410576    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 15:28:22.413608 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:28:22.413576    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8e7f05_26f5_4e04_9252_7133db41dbbf.slice/crio-187abdbbf5b303ac93ad1fb29dc50d744fee6be205d06c384062fa1afb8e626c WatchSource:0}: Error finding container 187abdbbf5b303ac93ad1fb29dc50d744fee6be205d06c384062fa1afb8e626c: Status 404 returned error can't find the container with id 187abdbbf5b303ac93ad1fb29dc50d744fee6be205d06c384062fa1afb8e626c
Jul 09 15:28:22.886708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.886676    2578 generic.go:356] "Generic (PLEG): container finished" podID="9a8e7f05-26f5-4e04-9252-7133db41dbbf" containerID="ba34e80b9d2be4456eea9b90a2c5bff85be77dce9f43309437c4bb18edf6ac03" exitCode=0
Jul 09 15:28:22.887374 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.886737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerDied","Data":"ba34e80b9d2be4456eea9b90a2c5bff85be77dce9f43309437c4bb18edf6ac03"}
Jul 09 15:28:22.887374 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:22.886855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"187abdbbf5b303ac93ad1fb29dc50d744fee6be205d06c384062fa1afb8e626c"}
Jul 09 15:28:23.895281 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"6a53aeb9f8475e189c094cf9dd9a1b2c419e1104ecea99512d630f473bd5598e"}
Jul 09 15:28:23.895693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895341    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"6b399e1f76d9f3809ea9bd295da7560b292daa35ba00ea94f36f8069293b9209"}
Jul 09 15:28:23.895693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"4578f0795334b8b42d2fc9a2f9fe7f4ca4992c5797e1b8637782ad7ceedd6c04"}
Jul 09 15:28:23.895693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895392    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"ec62aab3d6979915eb11232f9e5d88dc57579b680cf9c4eee421f408ac93ebd3"}
Jul 09 15:28:23.895693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895415    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"d19915d109dd2942519a615a35e47077ed84c93f129f7a22bfdc43d947df0dba"}
Jul 09 15:28:23.895693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.895438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9a8e7f05-26f5-4e04-9252-7133db41dbbf","Type":"ContainerStarted","Data":"0e29fd0acb5e315ae067f1e1e83296b2b26af05a1508735f0144ddfbfa639828"}
Jul 09 15:28:23.931978 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:23.931910    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.91315344 podStartE2EDuration="2.931889594s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.736154ms" isStatefulPod=false podCreationTimestamp="2026-07-09 15:28:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:28:23.930002407 +0000 UTC m=+110.390493415" watchObservedRunningTime="2026-07-09 15:28:23.931889594 +0000 UTC m=+110.392380592"
Jul 09 15:28:24.616311 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:24.616274    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:24.616473 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:24.616358    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:24.620916 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:24.620885    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:24.904423 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:24.904348    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:28:24.962354 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:24.962318    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:28:27.267523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:27.267482    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:28:49.989325 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:49.989264    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5946fd44fb-d5kpm" podUID="d8cfff4c-fdd2-419b-915f-2b188317ce19" containerName="console" containerID="cri-o://6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e" gracePeriod=15
Jul 09 15:28:50.233137 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.233108    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:28:50.338593 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338560    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.338780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338635    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.338780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338667    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.338780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338684    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-svh9t\" (UniqueName: \"kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.338780 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338706    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.339012 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.339012 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.338826    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle\") pod \"d8cfff4c-fdd2-419b-915f-2b188317ce19\" (UID: \"d8cfff4c-fdd2-419b-915f-2b188317ce19\") "
Jul 09 15:28:50.339114 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.339054    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:50.339185 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.339155    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca" (OuterVolumeSpecName: "service-ca") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:50.339293 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.339170    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config" (OuterVolumeSpecName: "console-config") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:50.339366 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.339332    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:28:50.340882 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.340858    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:50.341508 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.341480    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:28:50.341508 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.341486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t" (OuterVolumeSpecName: "kube-api-access-svh9t") pod "d8cfff4c-fdd2-419b-915f-2b188317ce19" (UID: "d8cfff4c-fdd2-419b-915f-2b188317ce19"). InnerVolumeSpecName "kube-api-access-svh9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:28:50.440120 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440093    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-service-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440120 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440119    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-oauth-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440129    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svh9t\" (UniqueName: \"kubernetes.io/projected/d8cfff4c-fdd2-419b-915f-2b188317ce19-kube-api-access-svh9t\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440139    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440147    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d8cfff4c-fdd2-419b-915f-2b188317ce19-console-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440155    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:50.440261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:50.440163    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d8cfff4c-fdd2-419b-915f-2b188317ce19-oauth-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:28:51.018799 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.018766    2578 generic.go:356] "Generic (PLEG): container finished" podID="d8cfff4c-fdd2-419b-915f-2b188317ce19" containerID="6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e" exitCode=0
Jul 09 15:28:51.019231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.018808    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5946fd44fb-d5kpm"
Jul 09 15:28:51.019231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.018824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5946fd44fb-d5kpm" event={"ID":"d8cfff4c-fdd2-419b-915f-2b188317ce19","Type":"ContainerDied","Data":"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"}
Jul 09 15:28:51.019231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.018865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5946fd44fb-d5kpm" event={"ID":"d8cfff4c-fdd2-419b-915f-2b188317ce19","Type":"ContainerDied","Data":"45f8c1d100afa718127e5802a2fc37134ea4f74668e83ed44036739ea57dca1d"}
Jul 09 15:28:51.019231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.018884    2578 scope.go:117] "RemoveContainer" containerID="6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"
Jul 09 15:28:51.027546 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.027527    2578 scope.go:117] "RemoveContainer" containerID="6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"
Jul 09 15:28:51.027800 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:28:51.027778    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e\": container with ID starting with 6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e not found: ID does not exist" containerID="6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"
Jul 09 15:28:51.027897 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.027807    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e"} err="failed to get container status \"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e\": rpc error: code = NotFound desc = could not find container \"6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e\": container with ID starting with 6023994a86e263eb500df31d743859eb55cb04fdf1f1cee68e720141d9ba453e not found: ID does not exist"
Jul 09 15:28:51.042495 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.042467    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:28:51.044645 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:51.044625    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5946fd44fb-d5kpm"]
Jul 09 15:28:52.104140 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:28:52.104104    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8cfff4c-fdd2-419b-915f-2b188317ce19" path="/var/lib/kubelet/pods/d8cfff4c-fdd2-419b-915f-2b188317ce19/volumes"
Jul 09 15:29:22.267435 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:29:22.267397    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:29:22.284236 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:29:22.284208    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:29:23.159901 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:29:23.159876    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 15:30:18.013665 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.013632    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-jqvf4"]
Jul 09 15:30:18.014094 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.013990    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8cfff4c-fdd2-419b-915f-2b188317ce19" containerName="console"
Jul 09 15:30:18.014094 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.014001    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8cfff4c-fdd2-419b-915f-2b188317ce19" containerName="console"
Jul 09 15:30:18.014094 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.014051    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8cfff4c-fdd2-419b-915f-2b188317ce19" containerName="console"
Jul 09 15:30:18.016218 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.016199    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.019104 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.019085    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\""
Jul 09 15:30:18.019210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.019127    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\""
Jul 09 15:30:18.019292 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.019275    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-h4k6m\""
Jul 09 15:30:18.020345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.020326    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\""
Jul 09 15:30:18.020430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.020357    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\""
Jul 09 15:30:18.036003 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.035975    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-jqvf4"]
Jul 09 15:30:18.186206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.186171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.186206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.186210    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfq6s\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-kube-api-access-gfq6s\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.286845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.286737    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.287360 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:30:18.287050    2578 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found
Jul 09 15:30:18.287360 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:30:18.287088    2578 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-jqvf4: secret "keda-admission-webhooks-certs" not found
Jul 09 15:30:18.287360 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.287105    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfq6s\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-kube-api-access-gfq6s\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.287360 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:30:18.287175    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates podName:201b05b4-6ede-43c7-be42-73dbff1ac7a6 nodeName:}" failed. No retries permitted until 2026-07-09 15:30:18.78714971 +0000 UTC m=+225.247640700 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates") pod "keda-admission-5fbd5c4644-jqvf4" (UID: "201b05b4-6ede-43c7-be42-73dbff1ac7a6") : secret "keda-admission-webhooks-certs" not found
Jul 09 15:30:18.300371 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.300346    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfq6s\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-kube-api-access-gfq6s\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.791717 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.791682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.794347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.794329    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/201b05b4-6ede-43c7-be42-73dbff1ac7a6-certificates\") pod \"keda-admission-5fbd5c4644-jqvf4\" (UID: \"201b05b4-6ede-43c7-be42-73dbff1ac7a6\") " pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:18.926103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:18.926071    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:19.055121 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:19.055101    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-jqvf4"]
Jul 09 15:30:19.056980 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:30:19.056946    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201b05b4_6ede_43c7_be42_73dbff1ac7a6.slice/crio-e2ffb4e5bb342c621d8cdfaa129c18d6b54624915aa37e9c29fe12c6a228736d WatchSource:0}: Error finding container e2ffb4e5bb342c621d8cdfaa129c18d6b54624915aa37e9c29fe12c6a228736d: Status 404 returned error can't find the container with id e2ffb4e5bb342c621d8cdfaa129c18d6b54624915aa37e9c29fe12c6a228736d
Jul 09 15:30:19.363013 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:19.362939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4" event={"ID":"201b05b4-6ede-43c7-be42-73dbff1ac7a6","Type":"ContainerStarted","Data":"e2ffb4e5bb342c621d8cdfaa129c18d6b54624915aa37e9c29fe12c6a228736d"}
Jul 09 15:30:22.377525 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:22.377489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4" event={"ID":"201b05b4-6ede-43c7-be42-73dbff1ac7a6","Type":"ContainerStarted","Data":"ddf4ea88e17ac4ff34d65bb9ab42bc4ec99d52386d237026ece828e0dbda8c1b"}
Jul 09 15:30:22.377946 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:22.377619    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:30:22.405949 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:22.405904    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4" podStartSLOduration=2.790474282 podStartE2EDuration="5.405892144s" totalImagesPullingTime="2.615417862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:30:22.403550929 +0000 UTC m=+228.864041927" watchObservedRunningTime="2026-07-09 15:30:22.405892144 +0000 UTC m=+228.866383141"
Jul 09 15:30:43.383983 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:30:43.383951    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-jqvf4"
Jul 09 15:31:33.931260 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:31:33.931227    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:31:33.936664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:31:33.936641    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:31:33.941877 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:31:33.941857    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 15:32:39.316106 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.316070    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bb7d6d4c9-8g7r2"]
Jul 09 15:32:39.319577 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.319555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.335979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.335958    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb7d6d4c9-8g7r2"]
Jul 09 15:32:39.492176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-trusted-ca-bundle\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-service-ca\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-oauth-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-oauth-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492494 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54lqt\" (UniqueName: \"kubernetes.io/projected/8dd56f4a-e51e-493a-a02c-8c62833fed66-kube-api-access-54lqt\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.492528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.492498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.592973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.592897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.592973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.592929    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-trusted-ca-bundle\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.592973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.592953    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-service-ca\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.592973 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.592974    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-oauth-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.592996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-oauth-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54lqt\" (UniqueName: \"kubernetes.io/projected/8dd56f4a-e51e-493a-a02c-8c62833fed66-kube-api-access-54lqt\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593876 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-service-ca\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593977 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593887    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593977 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593854    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-trusted-ca-bundle\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.593977 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.593934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8dd56f4a-e51e-493a-a02c-8c62833fed66-oauth-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.595581 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.595555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-oauth-config\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.595684 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.595560    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8dd56f4a-e51e-493a-a02c-8c62833fed66-console-serving-cert\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.603630 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.603610    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54lqt\" (UniqueName: \"kubernetes.io/projected/8dd56f4a-e51e-493a-a02c-8c62833fed66-kube-api-access-54lqt\") pod \"console-6bb7d6d4c9-8g7r2\" (UID: \"8dd56f4a-e51e-493a-a02c-8c62833fed66\") " pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.628668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.628645    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:39.764585 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.764562    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bb7d6d4c9-8g7r2"]
Jul 09 15:32:39.766464 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:32:39.766440    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dd56f4a_e51e_493a_a02c_8c62833fed66.slice/crio-6eee1af900125d032e8aaa8a47447b7ad0d4310e72375720ac775609da50fb34 WatchSource:0}: Error finding container 6eee1af900125d032e8aaa8a47447b7ad0d4310e72375720ac775609da50fb34: Status 404 returned error can't find the container with id 6eee1af900125d032e8aaa8a47447b7ad0d4310e72375720ac775609da50fb34
Jul 09 15:32:39.929695 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.929605    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb7d6d4c9-8g7r2" event={"ID":"8dd56f4a-e51e-493a-a02c-8c62833fed66","Type":"ContainerStarted","Data":"115fd0a90658d61a55b69089964a123e8c42cabad1d763ea192add45ab4c737c"}
Jul 09 15:32:39.929695 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.929662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bb7d6d4c9-8g7r2" event={"ID":"8dd56f4a-e51e-493a-a02c-8c62833fed66","Type":"ContainerStarted","Data":"6eee1af900125d032e8aaa8a47447b7ad0d4310e72375720ac775609da50fb34"}
Jul 09 15:32:39.952100 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:39.952049    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bb7d6d4c9-8g7r2" podStartSLOduration=0.952031867 podStartE2EDuration="952.031867ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:32:39.950948074 +0000 UTC m=+366.411439075" watchObservedRunningTime="2026-07-09 15:32:39.952031867 +0000 UTC m=+366.412522863"
Jul 09 15:32:49.628814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.628748    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:49.628814 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.628826    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:49.633255 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.633234    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:49.925413 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.925332    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-9zgrg"]
Jul 09 15:32:49.933152 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.933133    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-9zgrg"
Jul 09 15:32:49.936000 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.935978    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 15:32:49.936178 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.936008    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\""
Jul 09 15:32:49.936246 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.936215    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 15:32:49.936526 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.936510    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-dl444\""
Jul 09 15:32:49.938383 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.938358    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-9zgrg"]
Jul 09 15:32:49.972972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.972947    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bb7d6d4c9-8g7r2"
Jul 09 15:32:49.981887 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.981863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brx5z\" (UniqueName: \"kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:49.982133 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:49.982110    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.030664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.030634    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:32:50.083312 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.083280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.083468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.083362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brx5z\" (UniqueName: \"kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.083674 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.083658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.095129 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.095104    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brx5z\" (UniqueName: \"kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z\") pod \"s3-init-9zgrg\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") " pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.253929 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.253855    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-9zgrg"
Jul 09 15:32:50.385932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.385889    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-9zgrg"]
Jul 09 15:32:50.388160 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:32:50.388135    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod60cae34a_5134_4cef_8d4a_397c2675b335.slice/crio-922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2 WatchSource:0}: Error finding container 922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2: Status 404 returned error can't find the container with id 922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2
Jul 09 15:32:50.389908 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.389892    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:32:50.975678 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:50.975639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerStarted","Data":"922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2"}
Jul 09 15:32:54.995419 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:54.995373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerStarted","Data":"d68e46257b711a76af1005771f476d0d2d5608cefacd6a2e24e29080f836587d"}
Jul 09 15:32:56.000594 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:56.000562    2578 generic.go:356] "Generic (PLEG): container finished" podID="60cae34a-5134-4cef-8d4a-397c2675b335" containerID="d68e46257b711a76af1005771f476d0d2d5608cefacd6a2e24e29080f836587d" exitCode=0
Jul 09 15:32:56.000992 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:32:56.000610    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerDied","Data":"d68e46257b711a76af1005771f476d0d2d5608cefacd6a2e24e29080f836587d"}
Jul 09 15:33:02.030082 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:02.030044    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerStarted","Data":"748fc544f25312c7ccd8e68efb5794e38ac2a64781d6f188124232625a95f644"}
Jul 09 15:33:02.053819 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:02.053747    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-9zgrg" podStartSLOduration=2.614951827 podStartE2EDuration="13.053732498s" totalImagesPullingTime="8.892069909s" totalInitContainerRuntime="1.546710762s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:32:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:33:02.051135326 +0000 UTC m=+388.511626322" watchObservedRunningTime="2026-07-09 15:33:02.053732498 +0000 UTC m=+388.514223494"
Jul 09 15:33:15.052599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.052545    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d4ffcfbd8-z29kw" podUID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" containerName="console" containerID="cri-o://2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf" gracePeriod=15
Jul 09 15:33:15.297494 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.297469    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:33:15.409892 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.409806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.409892 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.409872    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.409911    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.409937    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dhgr\" (UniqueName: \"kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.409978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410075    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert\") pod \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\" (UID: \"4123ea31-d43c-4c08-910c-5ed7c96e45ed\") "
Jul 09 15:33:15.410331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410273    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:33:15.410331 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410311    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:33:15.410430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410390    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-service-ca\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.410430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410388    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config" (OuterVolumeSpecName: "console-config") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:33:15.410430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410407    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-trusted-ca-bundle\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.410597 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.410567    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:33:15.412251 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.412229    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr" (OuterVolumeSpecName: "kube-api-access-7dhgr") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "kube-api-access-7dhgr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:33:15.412596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.412570    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:33:15.412596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.412587    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4123ea31-d43c-4c08-910c-5ed7c96e45ed" (UID: "4123ea31-d43c-4c08-910c-5ed7c96e45ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:33:15.511683 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.511653    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-oauth-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.511683 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.511678    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-oauth-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.511683 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.511687    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dhgr\" (UniqueName: \"kubernetes.io/projected/4123ea31-d43c-4c08-910c-5ed7c96e45ed-kube-api-access-7dhgr\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.512001 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.511696    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-serving-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:15.512001 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:15.511705    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4123ea31-d43c-4c08-910c-5ed7c96e45ed-console-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:33:16.092210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.092174    2578 generic.go:356] "Generic (PLEG): container finished" podID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" containerID="2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf" exitCode=0
Jul 09 15:33:16.092652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.092220    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d4ffcfbd8-z29kw"
Jul 09 15:33:16.092652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.092239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4ffcfbd8-z29kw" event={"ID":"4123ea31-d43c-4c08-910c-5ed7c96e45ed","Type":"ContainerDied","Data":"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"}
Jul 09 15:33:16.092652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.092292    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d4ffcfbd8-z29kw" event={"ID":"4123ea31-d43c-4c08-910c-5ed7c96e45ed","Type":"ContainerDied","Data":"6863feb61c2f3e7c1293a25573c7696d5010cbd1784f79437af6cf39fc0e45a4"}
Jul 09 15:33:16.092652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.092325    2578 scope.go:117] "RemoveContainer" containerID="2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"
Jul 09 15:33:16.101745 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.101726    2578 scope.go:117] "RemoveContainer" containerID="2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"
Jul 09 15:33:16.102036 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:33:16.102008    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf\": container with ID starting with 2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf not found: ID does not exist" containerID="2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"
Jul 09 15:33:16.102105 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.102051    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf"} err="failed to get container status \"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf\": rpc error: code = NotFound desc = could not find container \"2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf\": container with ID starting with 2ef585a779c7c7c1f08c3850e9f175b85909df993a454b6f322e8ea688b903bf not found: ID does not exist"
Jul 09 15:33:16.117844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.117820    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:33:16.120334 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:16.120315    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d4ffcfbd8-z29kw"]
Jul 09 15:33:18.104014 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:33:18.103985    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" path="/var/lib/kubelet/pods/4123ea31-d43c-4c08-910c-5ed7c96e45ed/volumes"
Jul 09 15:34:06.296079 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:06.296047    2578 generic.go:356] "Generic (PLEG): container finished" podID="60cae34a-5134-4cef-8d4a-397c2675b335" containerID="748fc544f25312c7ccd8e68efb5794e38ac2a64781d6f188124232625a95f644" exitCode=0
Jul 09 15:34:06.296465 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:06.296120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerDied","Data":"748fc544f25312c7ccd8e68efb5794e38ac2a64781d6f188124232625a95f644"}
Jul 09 15:34:07.424884 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.424864    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-9zgrg"
Jul 09 15:34:07.562571 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.562493    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache\") pod \"60cae34a-5134-4cef-8d4a-397c2675b335\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") "
Jul 09 15:34:07.562571 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.562563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brx5z\" (UniqueName: \"kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z\") pod \"60cae34a-5134-4cef-8d4a-397c2675b335\" (UID: \"60cae34a-5134-4cef-8d4a-397c2675b335\") "
Jul 09 15:34:07.564863 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.564834    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z" (OuterVolumeSpecName: "kube-api-access-brx5z") pod "60cae34a-5134-4cef-8d4a-397c2675b335" (UID: "60cae34a-5134-4cef-8d4a-397c2675b335"). InnerVolumeSpecName "kube-api-access-brx5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:34:07.581288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.581261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache" (OuterVolumeSpecName: "model-cache") pod "60cae34a-5134-4cef-8d4a-397c2675b335" (UID: "60cae34a-5134-4cef-8d4a-397c2675b335"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:34:07.663102 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.663068    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60cae34a-5134-4cef-8d4a-397c2675b335-model-cache\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:07.663102 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:07.663095    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brx5z\" (UniqueName: \"kubernetes.io/projected/60cae34a-5134-4cef-8d4a-397c2675b335-kube-api-access-brx5z\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:08.305395 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:08.305358    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-9zgrg" event={"ID":"60cae34a-5134-4cef-8d4a-397c2675b335","Type":"ContainerDied","Data":"922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2"}
Jul 09 15:34:08.305395 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:08.305382    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-9zgrg"
Jul 09 15:34:08.305592 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:08.305405    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="922ae5b18472ba6af53b19428bd2605c71d853ac39141455e9c198606edda9c2"
Jul 09 15:34:09.101593 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101558    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101914    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60cae34a-5134-4cef-8d4a-397c2675b335" containerName="download-hf-model"
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101925    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60cae34a-5134-4cef-8d4a-397c2675b335" containerName="download-hf-model"
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101934    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" containerName="console"
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101940    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" containerName="console"
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101956    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60cae34a-5134-4cef-8d4a-397c2675b335" containerName="s3-init"
Jul 09 15:34:09.101971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.101962    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="60cae34a-5134-4cef-8d4a-397c2675b335" containerName="s3-init"
Jul 09 15:34:09.102233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.102008    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="60cae34a-5134-4cef-8d4a-397c2675b335" containerName="s3-init"
Jul 09 15:34:09.102233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.102018    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4123ea31-d43c-4c08-910c-5ed7c96e45ed" containerName="console"
Jul 09 15:34:09.105002 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.104986    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.107482 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.107458    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-dl444\""
Jul 09 15:34:09.107588 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.107544    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 15:34:09.108489 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.108466    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 09 15:34:09.108579 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.108560    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 15:34:09.114954 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.114926    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:09.176790 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.176738    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv6ps\" (UniqueName: \"kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.176930 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.176802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.278266 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.278234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv6ps\" (UniqueName: \"kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.278391 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.278346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.278691 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.278673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.289523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.289492    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv6ps\" (UniqueName: \"kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps\") pod \"seaweedfs-tls-custom-6644bf5d6c-nc477\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") " pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.414742 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.414640    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:09.542199 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:09.542182    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:09.544555 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:34:09.544523    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31b34c25_8c28_420a_86ae_1bc007bc0822.slice/crio-1c19367d00c1a2c7e5bb1232840f551791071a81f1ce3764927d4761b86869d8 WatchSource:0}: Error finding container 1c19367d00c1a2c7e5bb1232840f551791071a81f1ce3764927d4761b86869d8: Status 404 returned error can't find the container with id 1c19367d00c1a2c7e5bb1232840f551791071a81f1ce3764927d4761b86869d8
Jul 09 15:34:10.315835 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:10.315741    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" event={"ID":"31b34c25-8c28-420a-86ae-1bc007bc0822","Type":"ContainerStarted","Data":"1c19367d00c1a2c7e5bb1232840f551791071a81f1ce3764927d4761b86869d8"}
Jul 09 15:34:13.331922 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:13.331886    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" event={"ID":"31b34c25-8c28-420a-86ae-1bc007bc0822","Type":"ContainerStarted","Data":"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"}
Jul 09 15:34:13.350762 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:13.350712    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" podStartSLOduration=0.758609147 podStartE2EDuration="4.350697337s" totalImagesPullingTime="3.59208819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:13.349290947 +0000 UTC m=+459.809781955" watchObservedRunningTime="2026-07-09 15:34:13.350697337 +0000 UTC m=+459.811188334"
Jul 09 15:34:14.151075 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:14.151038    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:15.339696 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:15.339652    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" podUID="31b34c25-8c28-420a-86ae-1bc007bc0822" containerName="seaweedfs-tls-custom" containerID="cri-o://8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1" gracePeriod=30
Jul 09 15:34:43.874132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:43.874107    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:43.987210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:43.987124    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lv6ps\" (UniqueName: \"kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps\") pod \"31b34c25-8c28-420a-86ae-1bc007bc0822\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") "
Jul 09 15:34:43.987210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:43.987197    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data\") pod \"31b34c25-8c28-420a-86ae-1bc007bc0822\" (UID: \"31b34c25-8c28-420a-86ae-1bc007bc0822\") "
Jul 09 15:34:43.988440 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:43.988410    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data" (OuterVolumeSpecName: "data") pod "31b34c25-8c28-420a-86ae-1bc007bc0822" (UID: "31b34c25-8c28-420a-86ae-1bc007bc0822"). InnerVolumeSpecName "data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:34:43.989304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:43.989283    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps" (OuterVolumeSpecName: "kube-api-access-lv6ps") pod "31b34c25-8c28-420a-86ae-1bc007bc0822" (UID: "31b34c25-8c28-420a-86ae-1bc007bc0822"). InnerVolumeSpecName "kube-api-access-lv6ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:34:44.088506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.088475    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lv6ps\" (UniqueName: \"kubernetes.io/projected/31b34c25-8c28-420a-86ae-1bc007bc0822-kube-api-access-lv6ps\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:44.088506 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.088500    2578 reconciler_common.go:299] "Volume detached for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/31b34c25-8c28-420a-86ae-1bc007bc0822-data\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:44.459221 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.459187    2578 generic.go:356] "Generic (PLEG): container finished" podID="31b34c25-8c28-420a-86ae-1bc007bc0822" containerID="8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1" exitCode=0
Jul 09 15:34:44.459390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.459232    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"
Jul 09 15:34:44.459390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.459278    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" event={"ID":"31b34c25-8c28-420a-86ae-1bc007bc0822","Type":"ContainerDied","Data":"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"}
Jul 09 15:34:44.459390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.459334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-6644bf5d6c-nc477" event={"ID":"31b34c25-8c28-420a-86ae-1bc007bc0822","Type":"ContainerDied","Data":"1c19367d00c1a2c7e5bb1232840f551791071a81f1ce3764927d4761b86869d8"}
Jul 09 15:34:44.459390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.459360    2578 scope.go:117] "RemoveContainer" containerID="8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"
Jul 09 15:34:44.468891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.468874    2578 scope.go:117] "RemoveContainer" containerID="8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"
Jul 09 15:34:44.469159 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:34:44.469133    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1\": container with ID starting with 8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1 not found: ID does not exist" containerID="8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"
Jul 09 15:34:44.469210 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.469169    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1"} err="failed to get container status \"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1\": rpc error: code = NotFound desc = could not find container \"8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1\": container with ID starting with 8eb8eba904b2647008199b49f4d3fc6e0f750a8ca7e361384aaa6ca0ada2f1d1 not found: ID does not exist"
Jul 09 15:34:44.478900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.478881    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:44.480972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.480952    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/seaweedfs-tls-custom-6644bf5d6c-nc477"]
Jul 09 15:34:44.515932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.515893    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"]
Jul 09 15:34:44.516215 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.516203    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31b34c25-8c28-420a-86ae-1bc007bc0822" containerName="seaweedfs-tls-custom"
Jul 09 15:34:44.516256 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.516219    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="31b34c25-8c28-420a-86ae-1bc007bc0822" containerName="seaweedfs-tls-custom"
Jul 09 15:34:44.516299 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.516290    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="31b34c25-8c28-420a-86ae-1bc007bc0822" containerName="seaweedfs-tls-custom"
Jul 09 15:34:44.519551 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.519537    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.522015 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.521999    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom-artifact\""
Jul 09 15:34:44.522102 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.522000    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-dl444\""
Jul 09 15:34:44.522284 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.522270    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\""
Jul 09 15:34:44.522363 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.522349    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-custom\""
Jul 09 15:34:44.522627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.522615    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\""
Jul 09 15:34:44.528306 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.528281    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"]
Jul 09 15:34:44.591421 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.591393    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fa15aff0-6561-4d23-9940-0e961ec04b80-data\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.591594 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.591426    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgl87\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-kube-api-access-tgl87\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.591594 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.591462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.692444 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.692404    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fa15aff0-6561-4d23-9940-0e961ec04b80-data\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.692612 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.692449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgl87\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-kube-api-access-tgl87\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.692612 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.692500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.692825 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.692798    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/fa15aff0-6561-4d23-9940-0e961ec04b80-data\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.695091 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.695065    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"seaweedfs-tls-custom\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-seaweedfs-tls-custom\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.702343 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.702320    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgl87\" (UniqueName: \"kubernetes.io/projected/fa15aff0-6561-4d23-9940-0e961ec04b80-kube-api-access-tgl87\") pod \"seaweedfs-tls-custom-8b44d767d-2h4c8\" (UID: \"fa15aff0-6561-4d23-9940-0e961ec04b80\") " pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.829190 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.829160    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"
Jul 09 15:34:44.956435 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:44.956410    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-tls-custom-8b44d767d-2h4c8"]
Jul 09 15:34:44.958919 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:34:44.958894    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa15aff0_6561_4d23_9940_0e961ec04b80.slice/crio-3817e40280e09a0ee93e093aee42d1ca8ebc0975cbc00439cd29174cb411d739 WatchSource:0}: Error finding container 3817e40280e09a0ee93e093aee42d1ca8ebc0975cbc00439cd29174cb411d739: Status 404 returned error can't find the container with id 3817e40280e09a0ee93e093aee42d1ca8ebc0975cbc00439cd29174cb411d739
Jul 09 15:34:45.465836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.465801    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8" event={"ID":"fa15aff0-6561-4d23-9940-0e961ec04b80","Type":"ContainerStarted","Data":"918c5145018f5fa76ec3262e320aaa74c6cccf5cbff98ec9d5ce2d788b4a753e"}
Jul 09 15:34:45.465994 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.465849    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8" event={"ID":"fa15aff0-6561-4d23-9940-0e961ec04b80","Type":"ContainerStarted","Data":"3817e40280e09a0ee93e093aee42d1ca8ebc0975cbc00439cd29174cb411d739"}
Jul 09 15:34:45.485556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.485509    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-tls-custom-8b44d767d-2h4c8" podStartSLOduration=1.190525802 podStartE2EDuration="1.48549446s" totalImagesPullingTime="294.968658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:45.484280474 +0000 UTC m=+491.944771475" watchObservedRunningTime="2026-07-09 15:34:45.48549446 +0000 UTC m=+491.945985456"
Jul 09 15:34:45.798669 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.798633    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-custom-dnd58"]
Jul 09 15:34:45.802236 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.802215    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:45.814639 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.814609    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-dnd58"]
Jul 09 15:34:45.902477 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.902437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfh8f\" (UniqueName: \"kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:45.902644 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:45.902492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.003206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.003170    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfh8f\" (UniqueName: \"kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.003572 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.003241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.003617 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.003604    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.012954 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.012925    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfh8f\" (UniqueName: \"kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f\") pod \"s3-tls-init-custom-dnd58\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") " pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.104938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.104865    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31b34c25-8c28-420a-86ae-1bc007bc0822" path="/var/lib/kubelet/pods/31b34c25-8c28-420a-86ae-1bc007bc0822/volumes"
Jul 09 15:34:46.122712 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.122692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:46.254196 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.254147    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-custom-dnd58"]
Jul 09 15:34:46.256459 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:34:46.256427    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45d5c8e7_476e_4254_a986_aeded13a8180.slice/crio-60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0 WatchSource:0}: Error finding container 60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0: Status 404 returned error can't find the container with id 60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0
Jul 09 15:34:46.472616 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.472519    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerStarted","Data":"535d677a0180d16f38e9f96f69cea385251471b6daae8eb7707534f490c33a66"}
Jul 09 15:34:46.472616 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:46.472576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerStarted","Data":"60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0"}
Jul 09 15:34:48.484856 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:48.484823    2578 generic.go:356] "Generic (PLEG): container finished" podID="45d5c8e7-476e-4254-a986-aeded13a8180" containerID="535d677a0180d16f38e9f96f69cea385251471b6daae8eb7707534f490c33a66" exitCode=0
Jul 09 15:34:48.485303 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:48.484884    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerDied","Data":"535d677a0180d16f38e9f96f69cea385251471b6daae8eb7707534f490c33a66"}
Jul 09 15:34:49.491060 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:49.491025    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerStarted","Data":"f8980588f27045da637e546845bb3a4d2301951b65e799e116c0e6df2d36e3f4"}
Jul 09 15:34:49.512055 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:49.512003    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-custom-dnd58" podStartSLOduration=3.056037906 podStartE2EDuration="4.511986641s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.455948735s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:34:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:34:49.511512198 +0000 UTC m=+495.972003197" watchObservedRunningTime="2026-07-09 15:34:49.511986641 +0000 UTC m=+495.972477638"
Jul 09 15:34:55.520134 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:55.520101    2578 generic.go:356] "Generic (PLEG): container finished" podID="45d5c8e7-476e-4254-a986-aeded13a8180" containerID="f8980588f27045da637e546845bb3a4d2301951b65e799e116c0e6df2d36e3f4" exitCode=0
Jul 09 15:34:55.520489 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:55.520139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerDied","Data":"f8980588f27045da637e546845bb3a4d2301951b65e799e116c0e6df2d36e3f4"}
Jul 09 15:34:56.675173 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.675146    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:56.801662 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.801574    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache\") pod \"45d5c8e7-476e-4254-a986-aeded13a8180\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") "
Jul 09 15:34:56.801865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.801693    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfh8f\" (UniqueName: \"kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f\") pod \"45d5c8e7-476e-4254-a986-aeded13a8180\" (UID: \"45d5c8e7-476e-4254-a986-aeded13a8180\") "
Jul 09 15:34:56.803876 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.803848    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f" (OuterVolumeSpecName: "kube-api-access-gfh8f") pod "45d5c8e7-476e-4254-a986-aeded13a8180" (UID: "45d5c8e7-476e-4254-a986-aeded13a8180"). InnerVolumeSpecName "kube-api-access-gfh8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:34:56.823488 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.823454    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache" (OuterVolumeSpecName: "model-cache") pod "45d5c8e7-476e-4254-a986-aeded13a8180" (UID: "45d5c8e7-476e-4254-a986-aeded13a8180"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:34:56.902435 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.902404    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfh8f\" (UniqueName: \"kubernetes.io/projected/45d5c8e7-476e-4254-a986-aeded13a8180-kube-api-access-gfh8f\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:56.902435 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:56.902434    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/45d5c8e7-476e-4254-a986-aeded13a8180-model-cache\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:34:57.535663 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:57.535634    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-custom-dnd58"
Jul 09 15:34:57.535850 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:57.535632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-custom-dnd58" event={"ID":"45d5c8e7-476e-4254-a986-aeded13a8180","Type":"ContainerDied","Data":"60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0"}
Jul 09 15:34:57.535850 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:34:57.535742    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="60de1dba44568d31ed65a6010f66e33037161e1c01dd055ecda2923bce8c36e0"
Jul 09 15:35:04.747339 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747301    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-tls-init-serving-lp2hx"]
Jul 09 15:35:04.747708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747634    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d5c8e7-476e-4254-a986-aeded13a8180" containerName="download-hf-model"
Jul 09 15:35:04.747708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747646    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d5c8e7-476e-4254-a986-aeded13a8180" containerName="download-hf-model"
Jul 09 15:35:04.747708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747659    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45d5c8e7-476e-4254-a986-aeded13a8180" containerName="s3-tls-init-custom"
Jul 09 15:35:04.747708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747664    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="45d5c8e7-476e-4254-a986-aeded13a8180" containerName="s3-tls-init-custom"
Jul 09 15:35:04.747885 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.747724    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="45d5c8e7-476e-4254-a986-aeded13a8180" containerName="s3-tls-init-custom"
Jul 09 15:35:04.750570 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.750554    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.753566 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.753548    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"seaweedfs-tls-serving-artifact\""
Jul 09 15:35:04.759941 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.759918    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-lp2hx"]
Jul 09 15:35:04.880853 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.880811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bmkf\" (UniqueName: \"kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.881022 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.880927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.982207 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.982172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.982374 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.982253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bmkf\" (UniqueName: \"kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.982558 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.982535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:04.991557 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:04.991522    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bmkf\" (UniqueName: \"kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf\") pod \"s3-tls-init-serving-lp2hx\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") " pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:05.073865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:05.073831    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:05.205024 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:05.204992    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-tls-init-serving-lp2hx"]
Jul 09 15:35:05.208565 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:35:05.208537    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod026fff70_99da_49de_827d_8ff28ee7446c.slice/crio-6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead WatchSource:0}: Error finding container 6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead: Status 404 returned error can't find the container with id 6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead
Jul 09 15:35:05.570533 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:05.570503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerStarted","Data":"98e1bab9e6af32c0ea445791d5bc1efcdf5a89a720165745acdeed744c130493"}
Jul 09 15:35:05.570836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:05.570542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerStarted","Data":"6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead"}
Jul 09 15:35:07.580104 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:07.580071    2578 generic.go:356] "Generic (PLEG): container finished" podID="026fff70-99da-49de-827d-8ff28ee7446c" containerID="98e1bab9e6af32c0ea445791d5bc1efcdf5a89a720165745acdeed744c130493" exitCode=0
Jul 09 15:35:07.580484 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:07.580117    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerDied","Data":"98e1bab9e6af32c0ea445791d5bc1efcdf5a89a720165745acdeed744c130493"}
Jul 09 15:35:08.586426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:08.586389    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerStarted","Data":"e28225c3909adc0fc6458716d1c7dcd94755beaac93db104816595849c4c5936"}
Jul 09 15:35:08.605515 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:08.605467    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-tls-init-serving-lp2hx" podStartSLOduration=3.060997029 podStartE2EDuration="4.605453547s" totalImagesPullingTime="0s" totalInitContainerRuntime="1.544456518s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:35:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:35:08.6047322 +0000 UTC m=+515.065223199" watchObservedRunningTime="2026-07-09 15:35:08.605453547 +0000 UTC m=+515.065944544"
Jul 09 15:35:14.613274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:14.613243    2578 generic.go:356] "Generic (PLEG): container finished" podID="026fff70-99da-49de-827d-8ff28ee7446c" containerID="e28225c3909adc0fc6458716d1c7dcd94755beaac93db104816595849c4c5936" exitCode=0
Jul 09 15:35:14.613651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:14.613299    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerDied","Data":"e28225c3909adc0fc6458716d1c7dcd94755beaac93db104816595849c4c5936"}
Jul 09 15:35:15.756879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.756855    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:15.782500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.782479    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache\") pod \"026fff70-99da-49de-827d-8ff28ee7446c\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") "
Jul 09 15:35:15.782607 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.782529    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bmkf\" (UniqueName: \"kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf\") pod \"026fff70-99da-49de-827d-8ff28ee7446c\" (UID: \"026fff70-99da-49de-827d-8ff28ee7446c\") "
Jul 09 15:35:15.785139 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.785111    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf" (OuterVolumeSpecName: "kube-api-access-7bmkf") pod "026fff70-99da-49de-827d-8ff28ee7446c" (UID: "026fff70-99da-49de-827d-8ff28ee7446c"). InnerVolumeSpecName "kube-api-access-7bmkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:35:15.805060 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.805038    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache" (OuterVolumeSpecName: "model-cache") pod "026fff70-99da-49de-827d-8ff28ee7446c" (UID: "026fff70-99da-49de-827d-8ff28ee7446c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:35:15.883846 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.883790    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/026fff70-99da-49de-827d-8ff28ee7446c-model-cache\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:35:15.883846 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:15.883813    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bmkf\" (UniqueName: \"kubernetes.io/projected/026fff70-99da-49de-827d-8ff28ee7446c-kube-api-access-7bmkf\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:35:16.623010 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:16.622978    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-tls-init-serving-lp2hx" event={"ID":"026fff70-99da-49de-827d-8ff28ee7446c","Type":"ContainerDied","Data":"6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead"}
Jul 09 15:35:16.623010 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:16.622993    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-tls-init-serving-lp2hx"
Jul 09 15:35:16.623216 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:16.623020    2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6809d37b0b33c2c008dee0456ca67baa09cde1e47b97f0781b1824ddfeefaead"
Jul 09 15:35:28.542106 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542015    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:35:28.542523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542494    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026fff70-99da-49de-827d-8ff28ee7446c" containerName="download-hf-model"
Jul 09 15:35:28.542523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542508    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="026fff70-99da-49de-827d-8ff28ee7446c" containerName="download-hf-model"
Jul 09 15:35:28.542523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542519    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="026fff70-99da-49de-827d-8ff28ee7446c" containerName="s3-tls-init-serving"
Jul 09 15:35:28.542523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542524    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="026fff70-99da-49de-827d-8ff28ee7446c" containerName="s3-tls-init-serving"
Jul 09 15:35:28.542735 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.542593    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="026fff70-99da-49de-827d-8ff28ee7446c" containerName="s3-tls-init-serving"
Jul 09 15:35:28.544921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.544903    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.547461 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547439    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 15:35:28.547604 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547582    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 15:35:28.547604 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547592    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-kube-rbac-proxy-sar-config\""
Jul 09 15:35:28.547738 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547597    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 15:35:28.547816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547772    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-predictor-serving-cert\""
Jul 09 15:35:28.547876 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547861    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 15:35:28.547911 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.547886    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wc25c\""
Jul 09 15:35:28.559065 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.559039    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:35:28.593618 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.593584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.593816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.593636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.593816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.593699    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.593816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.593784    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p8dn\" (UniqueName: \"kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.593955 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.593818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p8dn\" (UniqueName: \"kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695334    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695371    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.695810 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.695788    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.696045 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.696026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.696132 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.696111    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.697926 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.697910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.705250 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.705221    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p8dn\" (UniqueName: \"kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn\") pod \"isvc-sklearn-batcher-predictor-7776d964d7-vx9c6\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.854593 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.854524    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:28.988207 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:28.988149    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:35:28.990289 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:35:28.990266    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1704513a_bd33_4185_8db1_eb0e5c729079.slice/crio-d400cc9fa8f2f7c17a3e4ac302daee1b210fd7ea41ea6e4db792914e55d2d0dd WatchSource:0}: Error finding container d400cc9fa8f2f7c17a3e4ac302daee1b210fd7ea41ea6e4db792914e55d2d0dd: Status 404 returned error can't find the container with id d400cc9fa8f2f7c17a3e4ac302daee1b210fd7ea41ea6e4db792914e55d2d0dd
Jul 09 15:35:29.680430 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:29.680393    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerStarted","Data":"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"}
Jul 09 15:35:29.680811 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:29.680470    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerStarted","Data":"d400cc9fa8f2f7c17a3e4ac302daee1b210fd7ea41ea6e4db792914e55d2d0dd"}
Jul 09 15:35:32.697953 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:32.697922    2578 generic.go:356] "Generic (PLEG): container finished" podID="1704513a-bd33-4185-8db1-eb0e5c729079" containerID="fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c" exitCode=0
Jul 09 15:35:32.698358 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:32.697972    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerDied","Data":"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"}
Jul 09 15:35:50.791620 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:50.791584    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerStarted","Data":"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"}
Jul 09 15:35:53.812211 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:53.812169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerStarted","Data":"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"}
Jul 09 15:35:56.830958 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:56.830919    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerStarted","Data":"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"}
Jul 09 15:35:56.831465 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:56.831025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:56.858529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:56.858482    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podStartSLOduration=2.283617667 podStartE2EDuration="28.858454979s" totalImagesPullingTime="23.075570988s" totalInitContainerRuntime="3.499266324s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:35:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:35:56.858186076 +0000 UTC m=+563.318677073" watchObservedRunningTime="2026-07-09 15:35:56.858454979 +0000 UTC m=+563.318945976"
Jul 09 15:35:57.835826 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:57.835792    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:57.836182 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:57.835858    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:57.837308 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:57.837266    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:35:57.837972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:57.837948    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:35:57.840721 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:57.840704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:35:58.840143 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:58.840103    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:35:58.840571 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:58.840412    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:35:59.844542 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:59.844505    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:35:59.844999 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:35:59.844780    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:09.844502 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:09.844453    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:09.844978 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:09.844874    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:19.845394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:19.845345    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:19.845820 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:19.845796    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:29.844793 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:29.844716    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:29.845319 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:29.845298    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:33.968288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:33.968253    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:36:33.968695 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:33.968302    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:36:39.844986 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:39.844943    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:39.845441 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:39.845404    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:49.844808 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:49.844722    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:49.845280 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:49.845171    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:36:59.844948 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:59.844850    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:36:59.845394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:36:59.845280    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:09.845529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:09.845499    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:09.845910 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:09.845698    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:23.656702 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.656666    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:37:23.657208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.657024    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" containerID="cri-o://5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d" gracePeriod=30
Jul 09 15:37:23.657208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.657036    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" containerID="cri-o://fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96" gracePeriod=30
Jul 09 15:37:23.657208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.657071    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" containerID="cri-o://ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6" gracePeriod=30
Jul 09 15:37:23.920470 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.920386    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:37:23.923208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.923188    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:23.926531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.926507    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-predictor-serving-cert\""
Jul 09 15:37:23.927934 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.927918    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-batcher-custom-kube-rbac-proxy-sar-config\""
Jul 09 15:37:23.948083 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:23.948059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:37:24.019365 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.019334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.019516 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.019377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.019516 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.019413    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.019516 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.019446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d224\" (UniqueName: \"kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.019623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.019521    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.120853 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.120824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.120997 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.120894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.120997 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.120939    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.120997 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.120971    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.121137 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.121108    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d224\" (UniqueName: \"kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.121293 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.121268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.121617 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.121596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.121701 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.121669    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.123389 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.123374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.131705 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.131687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d224\" (UniqueName: \"kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224\") pod \"isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") " pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.224176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.224094    2578 generic.go:356] "Generic (PLEG): container finished" podID="1704513a-bd33-4185-8db1-eb0e5c729079" containerID="ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6" exitCode=2
Jul 09 15:37:24.224176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.224134    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerDied","Data":"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"}
Jul 09 15:37:24.233305 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.233287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:24.372224 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:24.372131    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:37:24.375645 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:37:24.375619    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f85841f_fb69_4c8c_a71a_9e4ffbe32171.slice/crio-a1d8d1c07419ec47447233ab10149dc5c43bce81458fa27ea7239af6c900a263 WatchSource:0}: Error finding container a1d8d1c07419ec47447233ab10149dc5c43bce81458fa27ea7239af6c900a263: Status 404 returned error can't find the container with id a1d8d1c07419ec47447233ab10149dc5c43bce81458fa27ea7239af6c900a263
Jul 09 15:37:25.230367 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:25.230330    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerStarted","Data":"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"}
Jul 09 15:37:25.230771 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:25.230398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerStarted","Data":"a1d8d1c07419ec47447233ab10149dc5c43bce81458fa27ea7239af6c900a263"}
Jul 09 15:37:27.836936 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:27.836890    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:28.245995 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:28.245900    2578 generic.go:356] "Generic (PLEG): container finished" podID="1704513a-bd33-4185-8db1-eb0e5c729079" containerID="5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d" exitCode=0
Jul 09 15:37:28.245995 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:28.245957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerDied","Data":"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"}
Jul 09 15:37:28.247437 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:28.247414    2578 generic.go:356] "Generic (PLEG): container finished" podID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerID="3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135" exitCode=0
Jul 09 15:37:28.247525 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:28.247448    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerDied","Data":"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"}
Jul 09 15:37:29.255025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.254992    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerStarted","Data":"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"}
Jul 09 15:37:29.255407 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.255057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerStarted","Data":"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"}
Jul 09 15:37:29.255407 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.255083    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerStarted","Data":"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"}
Jul 09 15:37:29.255521 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.255449    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:29.255575 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.255527    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:29.255575 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.255568    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:29.256996 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.256964    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:37:29.257737 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.257708    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:29.282161 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.282120    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podStartSLOduration=2.761338917 podStartE2EDuration="6.282107449s" totalImagesPullingTime="0s" totalInitContainerRuntime="3.520768532s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:37:29.279121824 +0000 UTC m=+655.739612820" watchObservedRunningTime="2026-07-09 15:37:29.282107449 +0000 UTC m=+655.742598473"
Jul 09 15:37:29.844539 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.844495    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:37:29.844815 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:29.844790    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:30.259970 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:30.259871    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:37:30.260394 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:30.260373    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:32.836318 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:32.836272    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:35.264528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:35.264498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:37:35.265148 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:35.265120    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:37:35.265546 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:35.265522    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:37.836631 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:37.836584    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:37.837103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:37.836740    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:39.845082 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:39.845031    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:37:39.845445 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:39.845352    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:42.836163 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:42.836125    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:45.265055 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:45.265013    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:37:45.265531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:45.265505    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:47.836002 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:47.835962    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:49.845227 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:49.845175    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.32:8080: connect: connection refused"
Jul 09 15:37:49.845664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:49.845371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:49.845664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:49.845509    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:49.845664 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:49.845611    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:52.836329 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:52.836286    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.32:8643/healthz\": dial tcp 10.134.0.32:8643: connect: connection refused"
Jul 09 15:37:53.803263 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.803239    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:53.885918 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.885887    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config\") pod \"1704513a-bd33-4185-8db1-eb0e5c729079\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") "
Jul 09 15:37:53.886304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.885940    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p8dn\" (UniqueName: \"kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn\") pod \"1704513a-bd33-4185-8db1-eb0e5c729079\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") "
Jul 09 15:37:53.886304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.885968    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert\") pod \"1704513a-bd33-4185-8db1-eb0e5c729079\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") "
Jul 09 15:37:53.886304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.885996    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls\") pod \"1704513a-bd33-4185-8db1-eb0e5c729079\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") "
Jul 09 15:37:53.886304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.886019    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location\") pod \"1704513a-bd33-4185-8db1-eb0e5c729079\" (UID: \"1704513a-bd33-4185-8db1-eb0e5c729079\") "
Jul 09 15:37:53.886304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.886290    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1704513a-bd33-4185-8db1-eb0e5c729079" (UID: "1704513a-bd33-4185-8db1-eb0e5c729079"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:37:53.886501 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.886343    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1704513a-bd33-4185-8db1-eb0e5c729079" (UID: "1704513a-bd33-4185-8db1-eb0e5c729079"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:37:53.886501 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.886357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1704513a-bd33-4185-8db1-eb0e5c729079" (UID: "1704513a-bd33-4185-8db1-eb0e5c729079"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:37:53.888285 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.888262    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1704513a-bd33-4185-8db1-eb0e5c729079" (UID: "1704513a-bd33-4185-8db1-eb0e5c729079"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:37:53.888385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.888307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn" (OuterVolumeSpecName: "kube-api-access-4p8dn") pod "1704513a-bd33-4185-8db1-eb0e5c729079" (UID: "1704513a-bd33-4185-8db1-eb0e5c729079"). InnerVolumeSpecName "kube-api-access-4p8dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:37:53.986920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.986850    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:37:53.986920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.986876    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p8dn\" (UniqueName: \"kubernetes.io/projected/1704513a-bd33-4185-8db1-eb0e5c729079-kube-api-access-4p8dn\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:37:53.986920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.986885    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1704513a-bd33-4185-8db1-eb0e5c729079-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:37:53.986920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.986894    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1704513a-bd33-4185-8db1-eb0e5c729079-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:37:53.986920 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:53.986904    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1704513a-bd33-4185-8db1-eb0e5c729079-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:37:54.374443 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.374405    2578 generic.go:356] "Generic (PLEG): container finished" podID="1704513a-bd33-4185-8db1-eb0e5c729079" containerID="fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96" exitCode=0
Jul 09 15:37:54.374619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.374473    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"
Jul 09 15:37:54.374619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.374480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerDied","Data":"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"}
Jul 09 15:37:54.374619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.374551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6" event={"ID":"1704513a-bd33-4185-8db1-eb0e5c729079","Type":"ContainerDied","Data":"d400cc9fa8f2f7c17a3e4ac302daee1b210fd7ea41ea6e4db792914e55d2d0dd"}
Jul 09 15:37:54.374619 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.374581    2578 scope.go:117] "RemoveContainer" containerID="fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"
Jul 09 15:37:54.383207 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.383188    2578 scope.go:117] "RemoveContainer" containerID="ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"
Jul 09 15:37:54.390679 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.390661    2578 scope.go:117] "RemoveContainer" containerID="5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"
Jul 09 15:37:54.398061 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.398043    2578 scope.go:117] "RemoveContainer" containerID="fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"
Jul 09 15:37:54.398672 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.398652    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:37:54.403275 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.403253    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-predictor-7776d964d7-vx9c6"]
Jul 09 15:37:54.405545 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.405524    2578 scope.go:117] "RemoveContainer" containerID="fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"
Jul 09 15:37:54.405797 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:37:54.405777    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96\": container with ID starting with fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96 not found: ID does not exist" containerID="fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"
Jul 09 15:37:54.405855 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.405809    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96"} err="failed to get container status \"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96\": rpc error: code = NotFound desc = could not find container \"fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96\": container with ID starting with fca8e44f3b01bbdee5553066deb6a7eb25ca5dc832904b1be2c97de4e9e95c96 not found: ID does not exist"
Jul 09 15:37:54.405855 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.405832    2578 scope.go:117] "RemoveContainer" containerID="ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"
Jul 09 15:37:54.406068 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:37:54.406051    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6\": container with ID starting with ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6 not found: ID does not exist" containerID="ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"
Jul 09 15:37:54.406103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.406076    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6"} err="failed to get container status \"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6\": rpc error: code = NotFound desc = could not find container \"ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6\": container with ID starting with ff1533a2b961137334b36e94522990b6cbb90e2f62b67671a8075a11a7440bb6 not found: ID does not exist"
Jul 09 15:37:54.406103 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.406092    2578 scope.go:117] "RemoveContainer" containerID="5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"
Jul 09 15:37:54.406296 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:37:54.406281    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d\": container with ID starting with 5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d not found: ID does not exist" containerID="5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"
Jul 09 15:37:54.406339 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.406299    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d"} err="failed to get container status \"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d\": rpc error: code = NotFound desc = could not find container \"5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d\": container with ID starting with 5970e9b16a6261e6146639d390c0235ccfbd438b5274c83e7eb824fe662e2d4d not found: ID does not exist"
Jul 09 15:37:54.406339 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.406311    2578 scope.go:117] "RemoveContainer" containerID="fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"
Jul 09 15:37:54.406529 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:37:54.406515    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c\": container with ID starting with fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c not found: ID does not exist" containerID="fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"
Jul 09 15:37:54.406572 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:54.406534    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c"} err="failed to get container status \"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c\": rpc error: code = NotFound desc = could not find container \"fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c\": container with ID starting with fa811155370a0df0fa62ad3899f287d2312f2d02756d44f323ddbc6cf96b9a1c not found: ID does not exist"
Jul 09 15:37:55.265139 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:55.265087    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:37:55.265596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:55.265471    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:37:56.104367 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:37:56.104333    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" path="/var/lib/kubelet/pods/1704513a-bd33-4185-8db1-eb0e5c729079/volumes"
Jul 09 15:38:05.265101 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:05.265047    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:38:05.265562 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:05.265441    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:38:15.265265 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:15.265221    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:38:15.265707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:15.265685    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:38:25.265156 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:25.265048    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:38:25.265583 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:25.265548    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:38:35.265447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:35.265409    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:38:35.265953 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:35.265916    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:38:45.265848 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:45.265820    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:38:45.266260 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:45.266029    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:38:58.896027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:58.895990    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:38:58.896486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:58.896255    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" containerID="cri-o://b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763" gracePeriod=30
Jul 09 15:38:58.896486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:58.896262    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" containerID="cri-o://952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c" gracePeriod=30
Jul 09 15:38:58.896486 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:58.896259    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" containerID="cri-o://4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d" gracePeriod=30
Jul 09 15:38:59.002203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002177    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:38:59.002543 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002530    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent"
Jul 09 15:38:59.002591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002546    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent"
Jul 09 15:38:59.002591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002563    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy"
Jul 09 15:38:59.002591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002569    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy"
Jul 09 15:38:59.002591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002580    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container"
Jul 09 15:38:59.002591 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002585    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container"
Jul 09 15:38:59.002746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002594    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="storage-initializer"
Jul 09 15:38:59.002746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002600    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="storage-initializer"
Jul 09 15:38:59.002746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002665    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kube-rbac-proxy"
Jul 09 15:38:59.002746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002678    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="agent"
Jul 09 15:38:59.002746 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.002689    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1704513a-bd33-4185-8db1-eb0e5c729079" containerName="kserve-container"
Jul 09 15:38:59.006013 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.005993    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.008923 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.008905    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-predictor-serving-cert\""
Jul 09 15:38:59.009225 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.009209    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"message-dumper-kube-rbac-proxy-sar-config\""
Jul 09 15:38:59.026414 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.026387    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:38:59.147702 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.147626    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9w8n\" (UniqueName: \"kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.147702 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.147671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.147901 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.147720    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.248575 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.248537    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9w8n\" (UniqueName: \"kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.248800 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.248628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.248908 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.248880    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.249351 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.249322    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.251337 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.251317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.258345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.258324    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9w8n\" (UniqueName: \"kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n\") pod \"message-dumper-predictor-6575bcfdfd-htlbp\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") " pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.316954 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.316916    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:38:59.452350 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.452318    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:38:59.455331 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:38:59.455305    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06d9673b_7161_4a7b_9cad_62d32077f277.slice/crio-28c3b8232b208c729986510fe286249ad113cd0523205bf3172f9b28894f05f6 WatchSource:0}: Error finding container 28c3b8232b208c729986510fe286249ad113cd0523205bf3172f9b28894f05f6: Status 404 returned error can't find the container with id 28c3b8232b208c729986510fe286249ad113cd0523205bf3172f9b28894f05f6
Jul 09 15:38:59.457273 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.457257    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:38:59.676220 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.676130    2578 generic.go:356] "Generic (PLEG): container finished" podID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerID="4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d" exitCode=2
Jul 09 15:38:59.676220 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.676201    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerDied","Data":"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"}
Jul 09 15:38:59.677355 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:38:59.677334    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerStarted","Data":"28c3b8232b208c729986510fe286249ad113cd0523205bf3172f9b28894f05f6"}
Jul 09 15:39:00.260531 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:00.260494    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:01.690627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:01.690588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerStarted","Data":"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"}
Jul 09 15:39:01.691027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:01.690647    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerStarted","Data":"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"}
Jul 09 15:39:01.691027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:01.690671    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:39:01.712603 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:01.712547    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" podStartSLOduration=2.308064656 podStartE2EDuration="3.71253368s" totalImagesPullingTime="1.404469024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:01.710045443 +0000 UTC m=+748.170536439" watchObservedRunningTime="2026-07-09 15:39:01.71253368 +0000 UTC m=+748.173024733"
Jul 09 15:39:02.695039 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:02.695004    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:39:02.696472 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:02.696448    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:39:03.703496 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:03.703409    2578 generic.go:356] "Generic (PLEG): container finished" podID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerID="b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763" exitCode=0
Jul 09 15:39:03.703844 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:03.703486    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerDied","Data":"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"}
Jul 09 15:39:05.260159 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:05.260115    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:05.265502 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:05.265471    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:39:05.265910 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:05.265888    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:09.711975 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:09.711946    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:39:10.260652 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:10.260607    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:10.260872 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:10.260745    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:39:15.260811 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:15.260737    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:15.265018 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:15.264993    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:39:15.265395 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:15.265370    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:19.106668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.106634    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:39:19.110599 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.110582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.113262 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.113242    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-predictor-serving-cert\""
Jul 09 15:39:19.113511 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.113496    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-logger-kube-rbac-proxy-sar-config\""
Jul 09 15:39:19.126194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.126173    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:39:19.228872 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.228843    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b76w2\" (UniqueName: \"kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.229016 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.228886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.229016 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.228976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.229091 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.229018    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.229091 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.229051    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330259 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330270    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330396 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b76w2\" (UniqueName: \"kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330521 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.330778 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.330742    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.331050 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.331031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.331087 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.331031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.332932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.332916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.341107 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.341084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b76w2\" (UniqueName: \"kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2\") pod \"isvc-logger-predictor-5f586b99d8-45q5t\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") " pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.420304 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.420237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:19.562302 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.562277    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:39:19.564279 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:39:19.564251    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55ec3c91_7469_4e9d_92b2_f6d97b786e5f.slice/crio-559d58dc7b4ff7e5f8915b9660051b4e8d9ac967fda0ed22f3499b4b8eb2f0cf WatchSource:0}: Error finding container 559d58dc7b4ff7e5f8915b9660051b4e8d9ac967fda0ed22f3499b4b8eb2f0cf: Status 404 returned error can't find the container with id 559d58dc7b4ff7e5f8915b9660051b4e8d9ac967fda0ed22f3499b4b8eb2f0cf
Jul 09 15:39:19.781584 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.781553    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerStarted","Data":"21b75a2967dfaf38c9c0893a87e11f80c15a219120bd30575346c4732b25c3d1"}
Jul 09 15:39:19.781793 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:19.781624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerStarted","Data":"559d58dc7b4ff7e5f8915b9660051b4e8d9ac967fda0ed22f3499b4b8eb2f0cf"}
Jul 09 15:39:20.260734 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:20.260648    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:23.802627 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:23.802593    2578 generic.go:356] "Generic (PLEG): container finished" podID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerID="21b75a2967dfaf38c9c0893a87e11f80c15a219120bd30575346c4732b25c3d1" exitCode=0
Jul 09 15:39:23.803073 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:23.802674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerDied","Data":"21b75a2967dfaf38c9c0893a87e11f80c15a219120bd30575346c4732b25c3d1"}
Jul 09 15:39:24.811261 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.811200    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerStarted","Data":"fe8e2b90464acfccf3d8d2c571edecfaf4625ef4eda4b1efb7b3295b6ea04e18"}
Jul 09 15:39:24.811707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.811307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerStarted","Data":"d505c6dae14c8cde4b7213bd15fbb4e8da72cae42be063766532056d77f1a4e3"}
Jul 09 15:39:24.811707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.811348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerStarted","Data":"a31268f9212b6ba42494a2b8443ecb6f902a174a010f3e0965a47557971deb85"}
Jul 09 15:39:24.811707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.811680    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:24.811931 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.811854    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:24.813418 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.813388    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:24.839385 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:24.839342    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podStartSLOduration=1.75727417 podStartE2EDuration="5.839329662s" totalImagesPullingTime="0s" totalInitContainerRuntime="4.082055492s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:39:24.837407856 +0000 UTC m=+771.297898852" watchObservedRunningTime="2026-07-09 15:39:24.839329662 +0000 UTC m=+771.299820658"
Jul 09 15:39:25.261066 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.260976    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.33:8643/healthz\": dial tcp 10.134.0.33:8643: connect: connection refused"
Jul 09 15:39:25.265367 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.265336    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.33:5000: connect: connection refused"
Jul 09 15:39:25.265516 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.265500    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:39:25.265687 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.265664    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:25.265788 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.265774    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:39:25.817065 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.817023    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:25.817065 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.817056    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:25.818274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:25.818240    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:26.821519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:26.821471    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:26.821970 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:26.821818    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:29.047428 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.047401    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:39:29.126239 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126208    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls\") pod \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") "
Jul 09 15:39:29.126416 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126280    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location\") pod \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") "
Jul 09 15:39:29.126496 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126435    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config\") pod \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") "
Jul 09 15:39:29.126496 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126485    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7d224\" (UniqueName: \"kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224\") pod \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") "
Jul 09 15:39:29.126605 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126530    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert\") pod \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\" (UID: \"0f85841f-fb69-4c8c-a71a-9e4ffbe32171\") "
Jul 09 15:39:29.126605 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126552    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "0f85841f-fb69-4c8c-a71a-9e4ffbe32171" (UID: "0f85841f-fb69-4c8c-a71a-9e4ffbe32171"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:39:29.126790 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126776    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:39:29.126865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "0f85841f-fb69-4c8c-a71a-9e4ffbe32171" (UID: "0f85841f-fb69-4c8c-a71a-9e4ffbe32171"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:39:29.127031 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.126975    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "0f85841f-fb69-4c8c-a71a-9e4ffbe32171" (UID: "0f85841f-fb69-4c8c-a71a-9e4ffbe32171"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:39:29.128500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.128480    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "0f85841f-fb69-4c8c-a71a-9e4ffbe32171" (UID: "0f85841f-fb69-4c8c-a71a-9e4ffbe32171"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:39:29.128638 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.128613    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224" (OuterVolumeSpecName: "kube-api-access-7d224") pod "0f85841f-fb69-4c8c-a71a-9e4ffbe32171" (UID: "0f85841f-fb69-4c8c-a71a-9e4ffbe32171"). InnerVolumeSpecName "kube-api-access-7d224". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:39:29.227891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.227802    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:39:29.227891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.227838    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:39:29.227891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.227852    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:39:29.227891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.227867    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7d224\" (UniqueName: \"kubernetes.io/projected/0f85841f-fb69-4c8c-a71a-9e4ffbe32171-kube-api-access-7d224\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:39:29.838537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.838503    2578 generic.go:356] "Generic (PLEG): container finished" podID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerID="952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c" exitCode=0
Jul 09 15:39:29.838537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.838543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerDied","Data":"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"}
Jul 09 15:39:29.838743 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.838572    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"
Jul 09 15:39:29.838743 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.838596    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt" event={"ID":"0f85841f-fb69-4c8c-a71a-9e4ffbe32171","Type":"ContainerDied","Data":"a1d8d1c07419ec47447233ab10149dc5c43bce81458fa27ea7239af6c900a263"}
Jul 09 15:39:29.838743 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.838626    2578 scope.go:117] "RemoveContainer" containerID="952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"
Jul 09 15:39:29.847510 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.847493    2578 scope.go:117] "RemoveContainer" containerID="4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"
Jul 09 15:39:29.854939 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.854922    2578 scope.go:117] "RemoveContainer" containerID="b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"
Jul 09 15:39:29.862880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.862864    2578 scope.go:117] "RemoveContainer" containerID="3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"
Jul 09 15:39:29.864494 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.864473    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:39:29.868384 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.868365    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-batcher-custom-predictor-694d6c969d-lc4pt"]
Jul 09 15:39:29.872879 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.872860    2578 scope.go:117] "RemoveContainer" containerID="952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"
Jul 09 15:39:29.873142 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:39:29.873125    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c\": container with ID starting with 952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c not found: ID does not exist" containerID="952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"
Jul 09 15:39:29.873208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873149    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c"} err="failed to get container status \"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c\": rpc error: code = NotFound desc = could not find container \"952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c\": container with ID starting with 952bea33ab4b4b8d24bbfda6a189c1d11fc69f457eb9d9eb2f9eb28fe8b57a6c not found: ID does not exist"
Jul 09 15:39:29.873208 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873167    2578 scope.go:117] "RemoveContainer" containerID="4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"
Jul 09 15:39:29.873456 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:39:29.873439    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d\": container with ID starting with 4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d not found: ID does not exist" containerID="4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"
Jul 09 15:39:29.873504 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873462    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d"} err="failed to get container status \"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d\": rpc error: code = NotFound desc = could not find container \"4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d\": container with ID starting with 4a95e73de0af9f5cefc5430be2edafd3db68910c2e723a25333d97c80edcd03d not found: ID does not exist"
Jul 09 15:39:29.873504 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873477    2578 scope.go:117] "RemoveContainer" containerID="b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"
Jul 09 15:39:29.873695 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:39:29.873679    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763\": container with ID starting with b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763 not found: ID does not exist" containerID="b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"
Jul 09 15:39:29.873773 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873699    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763"} err="failed to get container status \"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763\": rpc error: code = NotFound desc = could not find container \"b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763\": container with ID starting with b9b7e97ec84ef433a962678738b42fa699ee17aa060b94d407bdaa44e071d763 not found: ID does not exist"
Jul 09 15:39:29.873773 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873711    2578 scope.go:117] "RemoveContainer" containerID="3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"
Jul 09 15:39:29.873958 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:39:29.873940    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135\": container with ID starting with 3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135 not found: ID does not exist" containerID="3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"
Jul 09 15:39:29.874000 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:29.873963    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135"} err="failed to get container status \"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135\": rpc error: code = NotFound desc = could not find container \"3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135\": container with ID starting with 3b05fc671970bd8cb5b7b5e6b185d3855d53865bbcce71494a2b55d8b2a73135 not found: ID does not exist"
Jul 09 15:39:30.105596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:30.105511    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" path="/var/lib/kubelet/pods/0f85841f-fb69-4c8c-a71a-9e4ffbe32171/volumes"
Jul 09 15:39:31.826029 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:31.826002    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:39:31.826643 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:31.826614    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:31.826902 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:31.826876    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:41.826529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:41.826482    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:41.827046 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:41.826873    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:39:51.826572 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:51.826526    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:39:51.827079 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:39:51.827055    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:40:01.827165 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:01.827063    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:40:01.827646 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:01.827624    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:40:11.826838 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:11.826780    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:40:11.827290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:11.827244    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:40:21.826979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:21.826942    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:40:21.827520 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:21.827496    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:40:31.826887 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:31.826832    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:40:31.827362 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:31.827337    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:40:41.827968 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:41.827938    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:40:41.828367 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:41.828212    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:40:54.117092 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.117063    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve-ci-e2e-test_message-dumper-predictor-6575bcfdfd-htlbp_06d9673b-7161-4a7b-9cad-62d32077f277/kserve-container/0.log"
Jul 09 15:40:54.365041 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.365006    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:40:54.365301 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.365272    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" containerID="cri-o://a31268f9212b6ba42494a2b8443ecb6f902a174a010f3e0965a47557971deb85" gracePeriod=30
Jul 09 15:40:54.365301 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.365283    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" containerID="cri-o://fe8e2b90464acfccf3d8d2c571edecfaf4625ef4eda4b1efb7b3295b6ea04e18" gracePeriod=30
Jul 09 15:40:54.365511 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.365298    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" containerID="cri-o://d505c6dae14c8cde4b7213bd15fbb4e8da72cae42be063766532056d77f1a4e3" gracePeriod=30
Jul 09 15:40:54.455501 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.455472    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456135    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456158    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456178    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="storage-initializer"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456186    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="storage-initializer"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456223    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456232    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456254    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456262    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456366    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kube-rbac-proxy"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456379    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="kserve-container"
Jul 09 15:40:54.456455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.456392    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0f85841f-fb69-4c8c-a71a-9e4ffbe32171" containerName="agent"
Jul 09 15:40:54.460050 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.460015    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.462542 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.462517    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-kube-rbac-proxy-sar-config\""
Jul 09 15:40:54.462789 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.462771    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v1-predictor-serving-cert\""
Jul 09 15:40:54.471315 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.470834    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:40:54.471315 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.471105    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kserve-container" containerID="cri-o://e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20" gracePeriod=30
Jul 09 15:40:54.471315 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.471178    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kube-rbac-proxy" containerID="cri-o://afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e" gracePeriod=30
Jul 09 15:40:54.474110 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.474046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:40:54.571522 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.571493    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvvzf\" (UniqueName: \"kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.571698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.571537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.571698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.571556    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.571698 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.571630    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.571836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.571717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672246 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvvzf\" (UniqueName: \"kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672421 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672421 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672278    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672421 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672306    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672421 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672345    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.672914 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.672890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.673069 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.673044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.673195 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.673178    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.674944 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.674921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.681932 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.681910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvvzf\" (UniqueName: \"kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf\") pod \"isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") " pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.709961 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.709940    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:40:54.771334 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.771300    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:40:54.773492 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.773471    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9w8n\" (UniqueName: \"kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n\") pod \"06d9673b-7161-4a7b-9cad-62d32077f277\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") "
Jul 09 15:40:54.773613 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.773512    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls\") pod \"06d9673b-7161-4a7b-9cad-62d32077f277\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") "
Jul 09 15:40:54.773613 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.773606    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config\") pod \"06d9673b-7161-4a7b-9cad-62d32077f277\" (UID: \"06d9673b-7161-4a7b-9cad-62d32077f277\") "
Jul 09 15:40:54.774327 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.774249    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "06d9673b-7161-4a7b-9cad-62d32077f277" (UID: "06d9673b-7161-4a7b-9cad-62d32077f277"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:40:54.779811 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.776904    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n" (OuterVolumeSpecName: "kube-api-access-t9w8n") pod "06d9673b-7161-4a7b-9cad-62d32077f277" (UID: "06d9673b-7161-4a7b-9cad-62d32077f277"). InnerVolumeSpecName "kube-api-access-t9w8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:40:54.781629 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.781559    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "06d9673b-7161-4a7b-9cad-62d32077f277" (UID: "06d9673b-7161-4a7b-9cad-62d32077f277"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:40:54.875696 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.875660    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9w8n\" (UniqueName: \"kubernetes.io/projected/06d9673b-7161-4a7b-9cad-62d32077f277-kube-api-access-t9w8n\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:40:54.875856 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.875700    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/06d9673b-7161-4a7b-9cad-62d32077f277-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:40:54.875856 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.875716    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/06d9673b-7161-4a7b-9cad-62d32077f277-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:40:54.923343 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:54.923317    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:40:54.925363 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:40:54.925339    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c88fe5_bbc0_45fa_8de8_f426711a6414.slice/crio-5a40096fbdcd8eb00f743b22935bf99561aec8b310bb122c369745facbbf0f71 WatchSource:0}: Error finding container 5a40096fbdcd8eb00f743b22935bf99561aec8b310bb122c369745facbbf0f71: Status 404 returned error can't find the container with id 5a40096fbdcd8eb00f743b22935bf99561aec8b310bb122c369745facbbf0f71
Jul 09 15:40:55.246678 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.246569    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerStarted","Data":"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"}
Jul 09 15:40:55.246678 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.246650    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerStarted","Data":"5a40096fbdcd8eb00f743b22935bf99561aec8b310bb122c369745facbbf0f71"}
Jul 09 15:40:55.249173 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.249142    2578 generic.go:356] "Generic (PLEG): container finished" podID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerID="d505c6dae14c8cde4b7213bd15fbb4e8da72cae42be063766532056d77f1a4e3" exitCode=2
Jul 09 15:40:55.249290 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.249205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerDied","Data":"d505c6dae14c8cde4b7213bd15fbb4e8da72cae42be063766532056d77f1a4e3"}
Jul 09 15:40:55.250715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250690    2578 generic.go:356] "Generic (PLEG): container finished" podID="06d9673b-7161-4a7b-9cad-62d32077f277" containerID="afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e" exitCode=2
Jul 09 15:40:55.250715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250711    2578 generic.go:356] "Generic (PLEG): container finished" podID="06d9673b-7161-4a7b-9cad-62d32077f277" containerID="e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20" exitCode=2
Jul 09 15:40:55.250900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerDied","Data":"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"}
Jul 09 15:40:55.250900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250780    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"
Jul 09 15:40:55.250900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250796    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerDied","Data":"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"}
Jul 09 15:40:55.250900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250817    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp" event={"ID":"06d9673b-7161-4a7b-9cad-62d32077f277","Type":"ContainerDied","Data":"28c3b8232b208c729986510fe286249ad113cd0523205bf3172f9b28894f05f6"}
Jul 09 15:40:55.250900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.250841    2578 scope.go:117] "RemoveContainer" containerID="afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"
Jul 09 15:40:55.259109 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.259091    2578 scope.go:117] "RemoveContainer" containerID="e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"
Jul 09 15:40:55.267918 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.267774    2578 scope.go:117] "RemoveContainer" containerID="afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"
Jul 09 15:40:55.268367 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:40:55.268339    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e\": container with ID starting with afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e not found: ID does not exist" containerID="afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"
Jul 09 15:40:55.268460 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.268377    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"} err="failed to get container status \"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e\": rpc error: code = NotFound desc = could not find container \"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e\": container with ID starting with afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e not found: ID does not exist"
Jul 09 15:40:55.268460 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.268399    2578 scope.go:117] "RemoveContainer" containerID="e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"
Jul 09 15:40:55.268793 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:40:55.268742    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20\": container with ID starting with e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20 not found: ID does not exist" containerID="e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"
Jul 09 15:40:55.268910 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.268802    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"} err="failed to get container status \"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20\": rpc error: code = NotFound desc = could not find container \"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20\": container with ID starting with e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20 not found: ID does not exist"
Jul 09 15:40:55.268910 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.268822    2578 scope.go:117] "RemoveContainer" containerID="afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"
Jul 09 15:40:55.269086 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.269065    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e"} err="failed to get container status \"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e\": rpc error: code = NotFound desc = could not find container \"afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e\": container with ID starting with afd2649519c2a4e0ba480ce122e00723b13a6f5bb49ffe3422c5ca8e54e0469e not found: ID does not exist"
Jul 09 15:40:55.269156 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.269088    2578 scope.go:117] "RemoveContainer" containerID="e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"
Jul 09 15:40:55.269330 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.269310    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20"} err="failed to get container status \"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20\": rpc error: code = NotFound desc = could not find container \"e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20\": container with ID starting with e6d64ff3dbe8495560fff112743414fae0dd4633586be89c3500d1fbbbf7aa20 not found: ID does not exist"
Jul 09 15:40:55.281985 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.281965    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:40:55.286360 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:55.286339    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/message-dumper-predictor-6575bcfdfd-htlbp"]
Jul 09 15:40:56.104635 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:56.104601    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" path="/var/lib/kubelet/pods/06d9673b-7161-4a7b-9cad-62d32077f277/volumes"
Jul 09 15:40:56.821618 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:56.821576    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:40:59.271422 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:59.271392    2578 generic.go:356] "Generic (PLEG): container finished" podID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerID="a31268f9212b6ba42494a2b8443ecb6f902a174a010f3e0965a47557971deb85" exitCode=0
Jul 09 15:40:59.271786 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:40:59.271430    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerDied","Data":"a31268f9212b6ba42494a2b8443ecb6f902a174a010f3e0965a47557971deb85"}
Jul 09 15:41:01.822622 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:01.822567    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:41:01.826937 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:01.826910    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:41:01.827320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:01.827299    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:41:06.822405 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:06.822362    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:41:06.822809 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:06.822496    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:41:11.822179 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:11.822138    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:41:11.826562 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:11.826521    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:41:11.826807 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:11.826790    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:41:16.822341 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:16.822294    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:41:21.822156 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:21.822114    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.35:8643/healthz\": dial tcp 10.134.0.35:8643: connect: connection refused"
Jul 09 15:41:21.826711 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:21.826671    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.35:8080: connect: connection refused"
Jul 09 15:41:21.826888 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:21.826852    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 15:41:21.826888 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:21.826874    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:41:21.827032 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:21.826989    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:41:24.389475 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.389449    2578 generic.go:356] "Generic (PLEG): container finished" podID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerID="fe8e2b90464acfccf3d8d2c571edecfaf4625ef4eda4b1efb7b3295b6ea04e18" exitCode=0
Jul 09 15:41:24.389824 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.389506    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerDied","Data":"fe8e2b90464acfccf3d8d2c571edecfaf4625ef4eda4b1efb7b3295b6ea04e18"}
Jul 09 15:41:24.526513 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.526490    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:41:24.631537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls\") pod \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") "
Jul 09 15:41:24.631537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631491    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location\") pod \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") "
Jul 09 15:41:24.631537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631528    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert\") pod \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") "
Jul 09 15:41:24.631832 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631558    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b76w2\" (UniqueName: \"kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2\") pod \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") "
Jul 09 15:41:24.631832 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631605    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config\") pod \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\" (UID: \"55ec3c91-7469-4e9d-92b2-f6d97b786e5f\") "
Jul 09 15:41:24.631989 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631954    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "55ec3c91-7469-4e9d-92b2-f6d97b786e5f" (UID: "55ec3c91-7469-4e9d-92b2-f6d97b786e5f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:41:24.632108 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.631986    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "55ec3c91-7469-4e9d-92b2-f6d97b786e5f" (UID: "55ec3c91-7469-4e9d-92b2-f6d97b786e5f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:41:24.632108 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.632054    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "55ec3c91-7469-4e9d-92b2-f6d97b786e5f" (UID: "55ec3c91-7469-4e9d-92b2-f6d97b786e5f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:41:24.633845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.633819    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "55ec3c91-7469-4e9d-92b2-f6d97b786e5f" (UID: "55ec3c91-7469-4e9d-92b2-f6d97b786e5f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:41:24.633845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.633820    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2" (OuterVolumeSpecName: "kube-api-access-b76w2") pod "55ec3c91-7469-4e9d-92b2-f6d97b786e5f" (UID: "55ec3c91-7469-4e9d-92b2-f6d97b786e5f"). InnerVolumeSpecName "kube-api-access-b76w2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:41:24.732651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.732580    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:41:24.732651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.732609    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:41:24.732651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.732618    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:41:24.732651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.732628    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:41:24.732651 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:24.732639    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b76w2\" (UniqueName: \"kubernetes.io/projected/55ec3c91-7469-4e9d-92b2-f6d97b786e5f-kube-api-access-b76w2\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:41:25.399828 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.399794    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t" event={"ID":"55ec3c91-7469-4e9d-92b2-f6d97b786e5f","Type":"ContainerDied","Data":"559d58dc7b4ff7e5f8915b9660051b4e8d9ac967fda0ed22f3499b4b8eb2f0cf"}
Jul 09 15:41:25.400194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.399841    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"
Jul 09 15:41:25.400194 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.399875    2578 scope.go:117] "RemoveContainer" containerID="fe8e2b90464acfccf3d8d2c571edecfaf4625ef4eda4b1efb7b3295b6ea04e18"
Jul 09 15:41:25.408383 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.408366    2578 scope.go:117] "RemoveContainer" containerID="d505c6dae14c8cde4b7213bd15fbb4e8da72cae42be063766532056d77f1a4e3"
Jul 09 15:41:25.415985 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.415966    2578 scope.go:117] "RemoveContainer" containerID="a31268f9212b6ba42494a2b8443ecb6f902a174a010f3e0965a47557971deb85"
Jul 09 15:41:25.423585 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.423567    2578 scope.go:117] "RemoveContainer" containerID="21b75a2967dfaf38c9c0893a87e11f80c15a219120bd30575346c4732b25c3d1"
Jul 09 15:41:25.424298 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.424279    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:41:25.428886 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:25.428862    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-logger-predictor-5f586b99d8-45q5t"]
Jul 09 15:41:26.104297 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:26.104259    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" path="/var/lib/kubelet/pods/55ec3c91-7469-4e9d-92b2-f6d97b786e5f/volumes"
Jul 09 15:41:27.412719 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:27.412687    2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerID="1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8" exitCode=0
Jul 09 15:41:27.413108 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:27.412783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerDied","Data":"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"}
Jul 09 15:41:34.004829 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:34.004796    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:41:34.004829 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:41:34.004813    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:43:46.105715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:46.105679    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerStarted","Data":"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"}
Jul 09 15:43:47.108243 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:47.108207    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerStarted","Data":"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"}
Jul 09 15:43:47.108681 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:47.108269    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:43:47.108681 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:47.108290    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:43:47.109819 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:47.109792    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 15:43:47.142347 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:47.142289    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podStartSLOduration=3.06473053 podStartE2EDuration="2m53.142271813s" totalImagesPullingTime="2m18.458696768s" totalInitContainerRuntime="31.618844515s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:43:47.141814482 +0000 UTC m=+1033.602305479" watchObservedRunningTime="2026-07-09 15:43:47.142271813 +0000 UTC m=+1033.602762812"
Jul 09 15:43:48.113053 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:48.113008    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 15:43:53.117125 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:53.117092    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:43:53.117692 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:43:53.117661    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.36:8080: connect: connection refused"
Jul 09 15:44:03.118455 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:03.118426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:44:04.734574 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:04.734534    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:44:04.734998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:04.734928    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container" containerID="cri-o://9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5" gracePeriod=30
Jul 09 15:44:04.735143 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:04.735092    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kube-rbac-proxy" containerID="cri-o://377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d" gracePeriod=30
Jul 09 15:44:05.191863 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.191822    2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerID="377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d" exitCode=2
Jul 09 15:44:05.192097 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.191900    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerDied","Data":"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"}
Jul 09 15:44:05.609139 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.609114    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:44:05.730950 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.730878    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config\") pod \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") "
Jul 09 15:44:05.730950 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.730925    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls\") pod \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") "
Jul 09 15:44:05.731122 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.730958    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location\") pod \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") "
Jul 09 15:44:05.731122 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.731040    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvvzf\" (UniqueName: \"kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf\") pod \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") "
Jul 09 15:44:05.731122 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.731079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert\") pod \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\" (UID: \"e0c88fe5-bbc0-45fa-8de8-f426711a6414\") "
Jul 09 15:44:05.731327 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.731258    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "e0c88fe5-bbc0-45fa-8de8-f426711a6414" (UID: "e0c88fe5-bbc0-45fa-8de8-f426711a6414"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:44:05.731494 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.731467    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "e0c88fe5-bbc0-45fa-8de8-f426711a6414" (UID: "e0c88fe5-bbc0-45fa-8de8-f426711a6414"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:44:05.733287 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.733259    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf" (OuterVolumeSpecName: "kube-api-access-hvvzf") pod "e0c88fe5-bbc0-45fa-8de8-f426711a6414" (UID: "e0c88fe5-bbc0-45fa-8de8-f426711a6414"). InnerVolumeSpecName "kube-api-access-hvvzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:44:05.733356 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.733307    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "e0c88fe5-bbc0-45fa-8de8-f426711a6414" (UID: "e0c88fe5-bbc0-45fa-8de8-f426711a6414"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:44:05.735655 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.735633    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "e0c88fe5-bbc0-45fa-8de8-f426711a6414" (UID: "e0c88fe5-bbc0-45fa-8de8-f426711a6414"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:44:05.832610 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.832572    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvvzf\" (UniqueName: \"kubernetes.io/projected/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-api-access-hvvzf\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:44:05.832610 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.832603    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:44:05.832610 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.832616    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:44:05.832892 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.832630    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/e0c88fe5-bbc0-45fa-8de8-f426711a6414-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:44:05.832892 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:05.832642    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/e0c88fe5-bbc0-45fa-8de8-f426711a6414-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:44:06.198554 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.198520    2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerID="9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5" exitCode=0
Jul 09 15:44:06.198694 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.198569    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerDied","Data":"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"}
Jul 09 15:44:06.198694 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.198587    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"
Jul 09 15:44:06.198694 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.198631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49" event={"ID":"e0c88fe5-bbc0-45fa-8de8-f426711a6414","Type":"ContainerDied","Data":"5a40096fbdcd8eb00f743b22935bf99561aec8b310bb122c369745facbbf0f71"}
Jul 09 15:44:06.198694 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.198660    2578 scope.go:117] "RemoveContainer" containerID="377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"
Jul 09 15:44:06.207816 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.207797    2578 scope.go:117] "RemoveContainer" containerID="9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"
Jul 09 15:44:06.215383 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.215365    2578 scope.go:117] "RemoveContainer" containerID="1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"
Jul 09 15:44:06.221469 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.221445    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:44:06.223933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.223846    2578 scope.go:117] "RemoveContainer" containerID="377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"
Jul 09 15:44:06.224237 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:44:06.224169    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d\": container with ID starting with 377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d not found: ID does not exist" containerID="377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"
Jul 09 15:44:06.224237 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.224205    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d"} err="failed to get container status \"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d\": rpc error: code = NotFound desc = could not find container \"377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d\": container with ID starting with 377785b7bf486e363503f406950f870581606b22cf261e5e396f6a5b6df4bd2d not found: ID does not exist"
Jul 09 15:44:06.224237 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.224228    2578 scope.go:117] "RemoveContainer" containerID="9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"
Jul 09 15:44:06.224517 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:44:06.224499    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5\": container with ID starting with 9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5 not found: ID does not exist" containerID="9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"
Jul 09 15:44:06.224578 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.224522    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5"} err="failed to get container status \"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5\": rpc error: code = NotFound desc = could not find container \"9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5\": container with ID starting with 9b603a06e2b9aa90d1e68e3417cf2fb123ea9f19a9b495a74d514f86a30205a5 not found: ID does not exist"
Jul 09 15:44:06.224578 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.224546    2578 scope.go:117] "RemoveContainer" containerID="1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"
Jul 09 15:44:06.224858 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:44:06.224836    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8\": container with ID starting with 1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8 not found: ID does not exist" containerID="1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"
Jul 09 15:44:06.224935 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.224866    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8"} err="failed to get container status \"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8\": rpc error: code = NotFound desc = could not find container \"1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8\": container with ID starting with 1aca3dcdeb823e2a4d87499df314cae07ded4106c655e797b319d83aa451c3a8 not found: ID does not exist"
Jul 09 15:44:06.225939 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:06.225918    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v1-predictor-7bbbbd79cd-mzp49"]
Jul 09 15:44:08.105495 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:08.105460    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" path="/var/lib/kubelet/pods/e0c88fe5-bbc0-45fa-8de8-f426711a6414/volumes"
Jul 09 15:44:10.053603 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053569    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053923    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="storage-initializer"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053935    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="storage-initializer"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053950    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053956    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053963    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053968    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053974    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053979    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053984    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="storage-initializer"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053989    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="storage-initializer"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053995    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.053999    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054007    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054016    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054029    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054037    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054059    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054067    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054128    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kserve-container"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054138    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054145    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kube-rbac-proxy"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054152    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="06d9673b-7161-4a7b-9cad-62d32077f277" containerName="kserve-container"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054157    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="55ec3c91-7469-4e9d-92b2-f6d97b786e5f" containerName="agent"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054163    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kserve-container"
Jul 09 15:44:10.054715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.054170    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c88fe5-bbc0-45fa-8de8-f426711a6414" containerName="kube-rbac-proxy"
Jul 09 15:44:10.078247 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.078208    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:44:10.078416 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.078331    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.080989 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.080963    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-predictor-serving-cert\""
Jul 09 15:44:10.081138 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081008    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 15:44:10.081138 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081052    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-kube-rbac-proxy-sar-config\""
Jul 09 15:44:10.081138 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081102    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wc25c\""
Jul 09 15:44:10.081288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081011    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 15:44:10.081288 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081117    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 15:44:10.081991 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.081974    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 15:44:10.272585 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.272552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.272813 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.272598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.272813 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.272614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.272813 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.272747    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmxdz\" (UniqueName: \"kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.272968 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.272823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374264 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374264 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374230    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374264 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374247    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmxdz\" (UniqueName: \"kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.374528 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:44:10.374380    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-predictor-serving-cert: secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 15:44:10.374528 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:44:10.374453    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls podName:ec61c573-6434-49c4-9aa3-5d754bfaedc8 nodeName:}" failed. No retries permitted until 2026-07-09 15:44:10.874431065 +0000 UTC m=+1057.334922056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls") pod "isvc-autogluon-v2-predictor-67b849d456-pg8q7" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8") : secret "isvc-autogluon-v2-predictor-serving-cert" not found
Jul 09 15:44:10.374708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.374593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.375059 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.375039    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.375101 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.375045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.383747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.383725    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmxdz\" (UniqueName: \"kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.878878 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.878837    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.881362 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.881340    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") pod \"isvc-autogluon-v2-predictor-67b849d456-pg8q7\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:10.989378 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:10.989342    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:11.144190 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:11.144167    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:44:11.146445 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:44:11.146407    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec61c573_6434_49c4_9aa3_5d754bfaedc8.slice/crio-4436b252f44ef2dfef7a35c8bfc32f81ca4aee32648453fb2dd79549a0bb3623 WatchSource:0}: Error finding container 4436b252f44ef2dfef7a35c8bfc32f81ca4aee32648453fb2dd79549a0bb3623: Status 404 returned error can't find the container with id 4436b252f44ef2dfef7a35c8bfc32f81ca4aee32648453fb2dd79549a0bb3623
Jul 09 15:44:11.221574 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:11.221544    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerStarted","Data":"4436b252f44ef2dfef7a35c8bfc32f81ca4aee32648453fb2dd79549a0bb3623"}
Jul 09 15:44:12.227002 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:12.226965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerStarted","Data":"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"}
Jul 09 15:44:41.353505 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:41.353463    2578 generic.go:356] "Generic (PLEG): container finished" podID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerID="0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5" exitCode=0
Jul 09 15:44:41.353924 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:41.353531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerDied","Data":"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"}
Jul 09 15:44:42.360948 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:42.360905    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerStarted","Data":"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"}
Jul 09 15:44:42.361439 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:42.360968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerStarted","Data":"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"}
Jul 09 15:44:42.361439 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:42.361156    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:42.382955 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:42.382893    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" podStartSLOduration=3.020313222 podStartE2EDuration="32.382871335s" totalImagesPullingTime="0s" totalInitContainerRuntime="29.362558113s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:44:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:44:42.381728281 +0000 UTC m=+1088.842219314" watchObservedRunningTime="2026-07-09 15:44:42.382871335 +0000 UTC m=+1088.843362333"
Jul 09 15:44:43.365348 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:43.365314    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:44:49.375712 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:44:49.375681    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:46:17.104328 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:17.104240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:46:20.261434 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:20.261396    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:46:20.261859 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:20.261615    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kserve-container" containerID="cri-o://7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea" gracePeriod=30
Jul 09 15:46:20.261859 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:20.261672    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kube-rbac-proxy" containerID="cri-o://6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1" gracePeriod=30
Jul 09 15:46:20.784230 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:20.784191    2578 generic.go:356] "Generic (PLEG): container finished" podID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerID="6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1" exitCode=2
Jul 09 15:46:20.784403 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:20.784269    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerDied","Data":"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"}
Jul 09 15:46:21.112391 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.112369    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:46:21.197836 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.197796    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert\") pod \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") "
Jul 09 15:46:21.197998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.197911    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") pod \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") "
Jul 09 15:46:21.197998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.197967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmxdz\" (UniqueName: \"kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz\") pod \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") "
Jul 09 15:46:21.197998 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.197994    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location\") pod \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") "
Jul 09 15:46:21.198171 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.198069    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config\") pod \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\" (UID: \"ec61c573-6434-49c4-9aa3-5d754bfaedc8\") "
Jul 09 15:46:21.198171 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.198106    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "ec61c573-6434-49c4-9aa3-5d754bfaedc8" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:46:21.198623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.198389    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:46:21.198623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.198416    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "ec61c573-6434-49c4-9aa3-5d754bfaedc8" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:46:21.200167 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.200137    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz" (OuterVolumeSpecName: "kube-api-access-gmxdz") pod "ec61c573-6434-49c4-9aa3-5d754bfaedc8" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8"). InnerVolumeSpecName "kube-api-access-gmxdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:46:21.200478 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.200455    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "ec61c573-6434-49c4-9aa3-5d754bfaedc8" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:46:21.203362 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.203338    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "ec61c573-6434-49c4-9aa3-5d754bfaedc8" (UID: "ec61c573-6434-49c4-9aa3-5d754bfaedc8"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:46:21.299039 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.299013    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/ec61c573-6434-49c4-9aa3-5d754bfaedc8-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:46:21.299039 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.299036    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmxdz\" (UniqueName: \"kubernetes.io/projected/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-api-access-gmxdz\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:46:21.299354 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.299045    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:46:21.299354 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.299055    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/ec61c573-6434-49c4-9aa3-5d754bfaedc8-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:46:21.790387 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.790347    2578 generic.go:356] "Generic (PLEG): container finished" podID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerID="7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea" exitCode=0
Jul 09 15:46:21.790556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.790411    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"
Jul 09 15:46:21.790556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.790446    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerDied","Data":"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"}
Jul 09 15:46:21.790556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.790494    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7" event={"ID":"ec61c573-6434-49c4-9aa3-5d754bfaedc8","Type":"ContainerDied","Data":"4436b252f44ef2dfef7a35c8bfc32f81ca4aee32648453fb2dd79549a0bb3623"}
Jul 09 15:46:21.790556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.790520    2578 scope.go:117] "RemoveContainer" containerID="6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"
Jul 09 15:46:21.799569 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.799550    2578 scope.go:117] "RemoveContainer" containerID="7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"
Jul 09 15:46:21.806871 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.806854    2578 scope.go:117] "RemoveContainer" containerID="0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"
Jul 09 15:46:21.815489 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.815464    2578 scope.go:117] "RemoveContainer" containerID="6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"
Jul 09 15:46:21.815806 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:46:21.815778    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1\": container with ID starting with 6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1 not found: ID does not exist" containerID="6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"
Jul 09 15:46:21.815880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.815818    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1"} err="failed to get container status \"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1\": rpc error: code = NotFound desc = could not find container \"6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1\": container with ID starting with 6a46bf020bead415aaca2ba01a90c71e1f968349957cc61d2fbd8d554bdcddd1 not found: ID does not exist"
Jul 09 15:46:21.815880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.815841    2578 scope.go:117] "RemoveContainer" containerID="7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"
Jul 09 15:46:21.817569 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:46:21.817540    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea\": container with ID starting with 7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea not found: ID does not exist" containerID="7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"
Jul 09 15:46:21.817647 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.817579    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea"} err="failed to get container status \"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea\": rpc error: code = NotFound desc = could not find container \"7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea\": container with ID starting with 7fcad029166916dffdd079ed8411d2121d9ad7ab4ff78f8210bef1ce00e61cea not found: ID does not exist"
Jul 09 15:46:21.817647 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.817597    2578 scope.go:117] "RemoveContainer" containerID="0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"
Jul 09 15:46:21.817985 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:46:21.817959    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5\": container with ID starting with 0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5 not found: ID does not exist" containerID="0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"
Jul 09 15:46:21.818047 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.817992    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5"} err="failed to get container status \"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5\": rpc error: code = NotFound desc = could not find container \"0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5\": container with ID starting with 0bd0de11ad8718771de782ca4745ab9188ff8d5dbd953d37610331f8aeb1eac5 not found: ID does not exist"
Jul 09 15:46:21.825123 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.825089    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:46:21.830059 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:21.830039    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-predictor-67b849d456-pg8q7"]
Jul 09 15:46:22.104643 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:22.104560    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" path="/var/lib/kubelet/pods/ec61c573-6434-49c4-9aa3-5d754bfaedc8/volumes"
Jul 09 15:46:25.466535 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466500    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466861    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kserve-container"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466872    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kserve-container"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466887    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="storage-initializer"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466893    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="storage-initializer"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466901    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kube-rbac-proxy"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466907    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kube-rbac-proxy"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466958    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kserve-container"
Jul 09 15:46:25.467027 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.466970    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec61c573-6434-49c4-9aa3-5d754bfaedc8" containerName="kube-rbac-proxy"
Jul 09 15:46:25.470839 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.470822    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.478588 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.478565    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-predictor-serving-cert\""
Jul 09 15:46:25.481258 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481241    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-variants-kube-rbac-proxy-sar-config\""
Jul 09 15:46:25.481375 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481301    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wc25c\""
Jul 09 15:46:25.481375 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481323    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 15:46:25.481590 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481573    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 15:46:25.481668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481623    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 15:46:25.481668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.481636    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 15:46:25.498101 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.498074    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:46:25.532621 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.532595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.532740 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.532662    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvx67\" (UniqueName: \"kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.532740 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.532698    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.532740 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.532717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.532890 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.532783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633175 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633149    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633338 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633206    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633338 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvx67\" (UniqueName: \"kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633338 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633338 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633338 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:46:25.633329    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-serving-cert: secret "isvc-autogluon-v2-variants-predictor-serving-cert" not found
Jul 09 15:46:25.633607 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:46:25.633404    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls podName:64ec2e29-b2ac-44d6-962e-2319499506ba nodeName:}" failed. No retries permitted until 2026-07-09 15:46:26.13338027 +0000 UTC m=+1192.593871257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls") pod "isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba") : secret "isvc-autogluon-v2-variants-predictor-serving-cert" not found
Jul 09 15:46:25.633807 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.633971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633955    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.634010 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.633979    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:25.647273 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:25.647245    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvx67\" (UniqueName: \"kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:26.138318 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.138285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:26.140722 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.140693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") pod \"isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:26.380390 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.380357    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:26.513108 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.513083    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:46:26.515047 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:46:26.515020    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ec2e29_b2ac_44d6_962e_2319499506ba.slice/crio-97bd7f265504901c3c555cc085e5385e716e5fdf3e726eb22a91277fe4790932 WatchSource:0}: Error finding container 97bd7f265504901c3c555cc085e5385e716e5fdf3e726eb22a91277fe4790932: Status 404 returned error can't find the container with id 97bd7f265504901c3c555cc085e5385e716e5fdf3e726eb22a91277fe4790932
Jul 09 15:46:26.822110 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.822066    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerStarted","Data":"ea35915247d0ba23dc29e10cb33af060fb77266f82fcfc8928967e678b659501"}
Jul 09 15:46:26.822291 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:26.822169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerStarted","Data":"97bd7f265504901c3c555cc085e5385e716e5fdf3e726eb22a91277fe4790932"}
Jul 09 15:46:34.038769 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:34.038723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:46:34.040875 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:34.040854    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:46:47.916921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:47.916884    2578 generic.go:356] "Generic (PLEG): container finished" podID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerID="ea35915247d0ba23dc29e10cb33af060fb77266f82fcfc8928967e678b659501" exitCode=0
Jul 09 15:46:47.917326 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:47.916953    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerDied","Data":"ea35915247d0ba23dc29e10cb33af060fb77266f82fcfc8928967e678b659501"}
Jul 09 15:46:48.922952 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:48.922920    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerStarted","Data":"5e11ad52f54e21b0d30712a20aa22e1f363b7bab0cc439d0100417ef6b90d308"}
Jul 09 15:46:48.923320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:48.922981    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerStarted","Data":"ffee815aae0447766c912649f7ac6053daf2e531b7e220dcc4a1867aa612fa41"}
Jul 09 15:46:48.923320 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:48.923177    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:48.947747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:48.947693    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" podStartSLOduration=3.5251974759999998 podStartE2EDuration="23.947678352s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.422480876s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:46:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:46:48.946502357 +0000 UTC m=+1215.406993365" watchObservedRunningTime="2026-07-09 15:46:48.947678352 +0000 UTC m=+1215.408169350"
Jul 09 15:46:49.930391 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:49.930361    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:46:55.943405 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:46:55.943371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:48:26.105519 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:26.105493    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:48:35.631299 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:35.631252    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:48:35.631668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:35.631508    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kserve-container" containerID="cri-o://ffee815aae0447766c912649f7ac6053daf2e531b7e220dcc4a1867aa612fa41" gracePeriod=30
Jul 09 15:48:35.631668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:35.631592    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kube-rbac-proxy" containerID="cri-o://5e11ad52f54e21b0d30712a20aa22e1f363b7bab0cc439d0100417ef6b90d308" gracePeriod=30
Jul 09 15:48:35.937903 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:35.937801    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.134.0.38:8643/healthz\": dial tcp 10.134.0.38:8643: connect: connection refused"
Jul 09 15:48:36.101176 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.101129    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kserve-container" probeResult="failure" output="Get \"http://10.134.0.38:8080/v2/models/isvc-autogluon-v2-variants/ready\": dial tcp 10.134.0.38:8080: connect: connection refused"
Jul 09 15:48:36.390126 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.390091    2578 generic.go:356] "Generic (PLEG): container finished" podID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerID="5e11ad52f54e21b0d30712a20aa22e1f363b7bab0cc439d0100417ef6b90d308" exitCode=2
Jul 09 15:48:36.390126 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.390122    2578 generic.go:356] "Generic (PLEG): container finished" podID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerID="ffee815aae0447766c912649f7ac6053daf2e531b7e220dcc4a1867aa612fa41" exitCode=0
Jul 09 15:48:36.390308 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.390168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerDied","Data":"5e11ad52f54e21b0d30712a20aa22e1f363b7bab0cc439d0100417ef6b90d308"}
Jul 09 15:48:36.390308 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.390231    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerDied","Data":"ffee815aae0447766c912649f7ac6053daf2e531b7e220dcc4a1867aa612fa41"}
Jul 09 15:48:36.480012 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.479987    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:48:36.658747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.658650    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config\") pod \"64ec2e29-b2ac-44d6-962e-2319499506ba\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") "
Jul 09 15:48:36.658747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.658703    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert\") pod \"64ec2e29-b2ac-44d6-962e-2319499506ba\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") "
Jul 09 15:48:36.659274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.658779    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location\") pod \"64ec2e29-b2ac-44d6-962e-2319499506ba\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") "
Jul 09 15:48:36.659274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.658854    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") pod \"64ec2e29-b2ac-44d6-962e-2319499506ba\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") "
Jul 09 15:48:36.659274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.658909    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvx67\" (UniqueName: \"kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67\") pod \"64ec2e29-b2ac-44d6-962e-2319499506ba\" (UID: \"64ec2e29-b2ac-44d6-962e-2319499506ba\") "
Jul 09 15:48:36.659274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.659140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "64ec2e29-b2ac-44d6-962e-2319499506ba" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:48:36.659274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.659145    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "64ec2e29-b2ac-44d6-962e-2319499506ba" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:48:36.661149 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.661124    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "64ec2e29-b2ac-44d6-962e-2319499506ba" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:48:36.661222 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.661149    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67" (OuterVolumeSpecName: "kube-api-access-xvx67") pod "64ec2e29-b2ac-44d6-962e-2319499506ba" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba"). InnerVolumeSpecName "kube-api-access-xvx67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:48:36.663679 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.663657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "64ec2e29-b2ac-44d6-962e-2319499506ba" (UID: "64ec2e29-b2ac-44d6-962e-2319499506ba"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:48:36.759590 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.759563    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:48:36.759590 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.759584    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/64ec2e29-b2ac-44d6-962e-2319499506ba-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:48:36.759590 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.759594    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/64ec2e29-b2ac-44d6-962e-2319499506ba-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:48:36.759806 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.759604    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/64ec2e29-b2ac-44d6-962e-2319499506ba-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:48:36.759806 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:36.759613    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvx67\" (UniqueName: \"kubernetes.io/projected/64ec2e29-b2ac-44d6-962e-2319499506ba-kube-api-access-xvx67\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:48:37.396865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.396841    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"
Jul 09 15:48:37.396865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.396846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5" event={"ID":"64ec2e29-b2ac-44d6-962e-2319499506ba","Type":"ContainerDied","Data":"97bd7f265504901c3c555cc085e5385e716e5fdf3e726eb22a91277fe4790932"}
Jul 09 15:48:37.397077 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.396918    2578 scope.go:117] "RemoveContainer" containerID="5e11ad52f54e21b0d30712a20aa22e1f363b7bab0cc439d0100417ef6b90d308"
Jul 09 15:48:37.410379 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.410360    2578 scope.go:117] "RemoveContainer" containerID="ffee815aae0447766c912649f7ac6053daf2e531b7e220dcc4a1867aa612fa41"
Jul 09 15:48:37.417974 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.417955    2578 scope.go:117] "RemoveContainer" containerID="ea35915247d0ba23dc29e10cb33af060fb77266f82fcfc8928967e678b659501"
Jul 09 15:48:37.425108 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.425085    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:48:37.428199 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:37.428180    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-variants-predictor-c6767c94c-t2fl5"]
Jul 09 15:48:38.104721 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:38.104682    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" path="/var/lib/kubelet/pods/64ec2e29-b2ac-44d6-962e-2319499506ba/volumes"
Jul 09 15:48:40.845092 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845061    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845381    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kserve-container"
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845391    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kserve-container"
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845399    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kube-rbac-proxy"
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845405    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kube-rbac-proxy"
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845424    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="storage-initializer"
Jul 09 15:48:40.845468 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845430    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="storage-initializer"
Jul 09 15:48:40.845687 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845489    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kube-rbac-proxy"
Jul 09 15:48:40.845687 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.845501    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64ec2e29-b2ac-44d6-962e-2319499506ba" containerName="kserve-container"
Jul 09 15:48:40.849882 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.849867    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.854815 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.854796    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-predictor-serving-cert\""
Jul 09 15:48:40.854940 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.854841    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 15:48:40.855341 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.855308    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 15:48:40.855341 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.855324    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 15:48:40.855341 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.855337    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wc25c\""
Jul 09 15:48:40.855529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.855342    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 15:48:40.855529 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.855356    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-v2-noslash-kube-rbac-proxy-sar-config\""
Jul 09 15:48:40.867024 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.867003    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:48:40.897236 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.897207    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.897359 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.897258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvzx8\" (UniqueName: \"kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.897359 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.897345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.897470 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.897389    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.897524 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.897460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998466 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998486    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998596 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998596 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:48:40.998585    2578 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-serving-cert: secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 09 15:48:40.998843 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvzx8\" (UniqueName: \"kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.998843 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:48:40.998670    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls podName:c0b343a2-0676-4742-a4cb-b2643b0da311 nodeName:}" failed. No retries permitted until 2026-07-09 15:48:41.498653204 +0000 UTC m=+1327.959144179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls") pod "isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311") : secret "isvc-autogluon-v2-noslash-predictor-serving-cert" not found
Jul 09 15:48:40.999021 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.998999    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.999230 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.999214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:40.999267 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:40.999241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:41.007809 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:41.007790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvzx8\" (UniqueName: \"kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:41.503345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:41.503311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:41.505788 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:41.505767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") pod \"isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") " pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:41.759795 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:41.759687    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:41.892689 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:41.892666    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:48:41.895379 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:48:41.895336    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0b343a2_0676_4742_a4cb_b2643b0da311.slice/crio-10b7013d100a878234143f7f950760a0db21596ef4a323160faff7e8c45e643c WatchSource:0}: Error finding container 10b7013d100a878234143f7f950760a0db21596ef4a323160faff7e8c45e643c: Status 404 returned error can't find the container with id 10b7013d100a878234143f7f950760a0db21596ef4a323160faff7e8c45e643c
Jul 09 15:48:42.421312 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:42.421268    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerStarted","Data":"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"}
Jul 09 15:48:42.421577 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:42.421347    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerStarted","Data":"10b7013d100a878234143f7f950760a0db21596ef4a323160faff7e8c45e643c"}
Jul 09 15:48:47.445168 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:47.445087    2578 generic.go:356] "Generic (PLEG): container finished" podID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerID="24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3" exitCode=0
Jul 09 15:48:47.445521 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:47.445160    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerDied","Data":"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"}
Jul 09 15:48:48.452375 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:48.452343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerStarted","Data":"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"}
Jul 09 15:48:48.452792 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:48.452408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerStarted","Data":"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"}
Jul 09 15:48:48.452792 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:48.452613    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:48.486491 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:48.480680    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" podStartSLOduration=3.272731123 podStartE2EDuration="8.480652072s" totalImagesPullingTime="0s" totalInitContainerRuntime="5.207920949s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:48:48.478834094 +0000 UTC m=+1334.939325093" watchObservedRunningTime="2026-07-09 15:48:48.480652072 +0000 UTC m=+1334.941143071"
Jul 09 15:48:49.458642 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:49.458604    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:48:55.469707 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:48:55.469635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:50:17.103874 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:17.103846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:50:20.906928 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:20.906897    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:50:20.907337 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:20.907153    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kserve-container" containerID="cri-o://53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970" gracePeriod=30
Jul 09 15:50:20.907337 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:20.907181    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kube-rbac-proxy" containerID="cri-o://756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb" gracePeriod=30
Jul 09 15:50:21.752668 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.752645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:50:21.883810 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883704    2578 generic.go:356] "Generic (PLEG): container finished" podID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerID="756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb" exitCode=2
Jul 09 15:50:21.883810 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883731    2578 generic.go:356] "Generic (PLEG): container finished" podID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerID="53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970" exitCode=0
Jul 09 15:50:21.883810 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883795    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerDied","Data":"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"}
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerDied","Data":"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"}
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883950    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c" event={"ID":"c0b343a2-0676-4742-a4cb-b2643b0da311","Type":"ContainerDied","Data":"10b7013d100a878234143f7f950760a0db21596ef4a323160faff7e8c45e643c"}
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.883978    2578 scope.go:117] "RemoveContainer" containerID="756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location\") pod \"c0b343a2-0676-4742-a4cb-b2643b0da311\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") "
Jul 09 15:50:21.884057 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884046    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") pod \"c0b343a2-0676-4742-a4cb-b2643b0da311\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") "
Jul 09 15:50:21.884382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config\") pod \"c0b343a2-0676-4742-a4cb-b2643b0da311\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") "
Jul 09 15:50:21.884382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884105    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert\") pod \"c0b343a2-0676-4742-a4cb-b2643b0da311\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") "
Jul 09 15:50:21.884382 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvzx8\" (UniqueName: \"kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8\") pod \"c0b343a2-0676-4742-a4cb-b2643b0da311\" (UID: \"c0b343a2-0676-4742-a4cb-b2643b0da311\") "
Jul 09 15:50:21.884534 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884398    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "c0b343a2-0676-4742-a4cb-b2643b0da311" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:50:21.884534 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.884486    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "c0b343a2-0676-4742-a4cb-b2643b0da311" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:50:21.886368 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.886343    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c0b343a2-0676-4742-a4cb-b2643b0da311" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:50:21.886493 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.886440    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8" (OuterVolumeSpecName: "kube-api-access-qvzx8") pod "c0b343a2-0676-4742-a4cb-b2643b0da311" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311"). InnerVolumeSpecName "kube-api-access-qvzx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:50:21.888963 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.888936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "c0b343a2-0676-4742-a4cb-b2643b0da311" (UID: "c0b343a2-0676-4742-a4cb-b2643b0da311"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:50:21.897939 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.897922    2578 scope.go:117] "RemoveContainer" containerID="53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"
Jul 09 15:50:21.905462 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.905435    2578 scope.go:117] "RemoveContainer" containerID="24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"
Jul 09 15:50:21.912781 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.912615    2578 scope.go:117] "RemoveContainer" containerID="756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"
Jul 09 15:50:21.913082 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:21.912914    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb\": container with ID starting with 756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb not found: ID does not exist" containerID="756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"
Jul 09 15:50:21.913082 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.912938    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"} err="failed to get container status \"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb\": rpc error: code = NotFound desc = could not find container \"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb\": container with ID starting with 756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb not found: ID does not exist"
Jul 09 15:50:21.913082 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.912954    2578 scope.go:117] "RemoveContainer" containerID="53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"
Jul 09 15:50:21.913256 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:21.913179    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970\": container with ID starting with 53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970 not found: ID does not exist" containerID="53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"
Jul 09 15:50:21.913256 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913203    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"} err="failed to get container status \"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970\": rpc error: code = NotFound desc = could not find container \"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970\": container with ID starting with 53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970 not found: ID does not exist"
Jul 09 15:50:21.913256 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913218    2578 scope.go:117] "RemoveContainer" containerID="24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"
Jul 09 15:50:21.913479 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:21.913456    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3\": container with ID starting with 24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3 not found: ID does not exist" containerID="24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"
Jul 09 15:50:21.913537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913487    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"} err="failed to get container status \"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3\": rpc error: code = NotFound desc = could not find container \"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3\": container with ID starting with 24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3 not found: ID does not exist"
Jul 09 15:50:21.913537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913503    2578 scope.go:117] "RemoveContainer" containerID="756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"
Jul 09 15:50:21.913745 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913723    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb"} err="failed to get container status \"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb\": rpc error: code = NotFound desc = could not find container \"756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb\": container with ID starting with 756ee3888d06d1db28680aaa10a1c070479ae733972e2df509a5d19f70acf5cb not found: ID does not exist"
Jul 09 15:50:21.913745 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.913766    2578 scope.go:117] "RemoveContainer" containerID="53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"
Jul 09 15:50:21.914018 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.914001    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970"} err="failed to get container status \"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970\": rpc error: code = NotFound desc = could not find container \"53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970\": container with ID starting with 53e2e7d9d6f7326cc30bae3086ba8d0757ad421a684386d897296f79ba512970 not found: ID does not exist"
Jul 09 15:50:21.914089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.914022    2578 scope.go:117] "RemoveContainer" containerID="24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"
Jul 09 15:50:21.914252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.914235    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3"} err="failed to get container status \"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3\": rpc error: code = NotFound desc = could not find container \"24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3\": container with ID starting with 24af09552c6cb91cbe47d42db8f1f806fb488b665c5e8a3b0b4b9442121151f3 not found: ID does not exist"
Jul 09 15:50:21.985790 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.985744    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c0b343a2-0676-4742-a4cb-b2643b0da311-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:21.985790 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.985789    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:21.985925 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.985799    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/c0b343a2-0676-4742-a4cb-b2643b0da311-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:21.985925 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.985809    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvzx8\" (UniqueName: \"kubernetes.io/projected/c0b343a2-0676-4742-a4cb-b2643b0da311-kube-api-access-qvzx8\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:21.985925 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:21.985817    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/c0b343a2-0676-4742-a4cb-b2643b0da311-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:22.204715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:22.204643    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:50:22.207747 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:22.207728    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-v2-noslash-predictor-9dcfbbb84-kjm4c"]
Jul 09 15:50:24.104492 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:24.104455    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" path="/var/lib/kubelet/pods/c0b343a2-0676-4742-a4cb-b2643b0da311/volumes"
Jul 09 15:50:26.207576 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207501    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207876    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kube-rbac-proxy"
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207889    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kube-rbac-proxy"
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207899    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="storage-initializer"
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207904    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="storage-initializer"
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207923    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kserve-container"
Jul 09 15:50:26.207933 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207932    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kserve-container"
Jul 09 15:50:26.208125 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.207989    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kube-rbac-proxy"
Jul 09 15:50:26.208125 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.208003    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0b343a2-0676-4742-a4cb-b2643b0da311" containerName="kserve-container"
Jul 09 15:50:26.212972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.212952    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.216393 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.216372    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\""
Jul 09 15:50:26.217352 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.217300    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-kube-rbac-proxy-sar-config\""
Jul 09 15:50:26.217500 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.217354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-wc25c\""
Jul 09 15:50:26.218471 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.217741    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\""
Jul 09 15:50:26.218471 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.218013    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\""
Jul 09 15:50:26.218471 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.218314    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-autogluon-ts-v1-predictor-serving-cert\""
Jul 09 15:50:26.218708 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.218563    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\""
Jul 09 15:50:26.233726 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.233707    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:26.325274 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.325236    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.325447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.325304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.325447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.325332    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.325447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.325388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqxs9\" (UniqueName: \"kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.325447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.325409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.425980 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.425948    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426146 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.425990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426146 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426032    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqxs9\" (UniqueName: \"kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426146 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426146 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426366 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426341    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426770 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.426883 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.426770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.428493 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.428477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.437013 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.436987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqxs9\" (UniqueName: \"kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9\") pod \"isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") " pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.525218 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.525187    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:26.665324 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.665288    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:26.673703 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:50:26.673675    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ad131b5_831f_4c6e_bb23_ade275d7f79f.slice/crio-c4656f5c4bd44440c2f9736b0418b80296213dd3c62ef65a162cfefe1e7d1480 WatchSource:0}: Error finding container c4656f5c4bd44440c2f9736b0418b80296213dd3c62ef65a162cfefe1e7d1480: Status 404 returned error can't find the container with id c4656f5c4bd44440c2f9736b0418b80296213dd3c62ef65a162cfefe1e7d1480
Jul 09 15:50:26.906800 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.906694    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerStarted","Data":"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"}
Jul 09 15:50:26.906800 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:26.906783    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerStarted","Data":"c4656f5c4bd44440c2f9736b0418b80296213dd3c62ef65a162cfefe1e7d1480"}
Jul 09 15:50:34.944951 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:34.944917    2578 generic.go:356] "Generic (PLEG): container finished" podID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerID="3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074" exitCode=0
Jul 09 15:50:34.944951 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:34.944955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerDied","Data":"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"}
Jul 09 15:50:35.951285 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:35.951242    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerStarted","Data":"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"}
Jul 09 15:50:35.951663 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:35.951326    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerStarted","Data":"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"}
Jul 09 15:50:35.951663 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:35.951560    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:35.977586 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:35.977535    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podStartSLOduration=2.196750706 podStartE2EDuration="9.977518064s" totalImagesPullingTime="0s" totalInitContainerRuntime="7.780767358s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:50:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:50:35.976404579 +0000 UTC m=+1442.436895577" watchObservedRunningTime="2026-07-09 15:50:35.977518064 +0000 UTC m=+1442.438009160"
Jul 09 15:50:36.958149 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:36.958113    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:36.961679 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:36.961644    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 09 15:50:37.960623 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:37.960585    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 09 15:50:42.964876 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:42.964849    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:42.965354 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:42.965329    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container" probeResult="failure" output="dial tcp 10.134.0.40:8080: connect: connection refused"
Jul 09 15:50:52.965971 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:52.965934    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:56.157986 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:56.157953    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:56.158409 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:56.158164    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container" containerID="cri-o://eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee" gracePeriod=30
Jul 09 15:50:56.158409 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:56.158194    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kube-rbac-proxy" containerID="cri-o://87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a" gracePeriod=30
Jul 09 15:50:57.007520 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.007492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:57.046665 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046633    2578 generic.go:356] "Generic (PLEG): container finished" podID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerID="87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a" exitCode=2
Jul 09 15:50:57.046665 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046658    2578 generic.go:356] "Generic (PLEG): container finished" podID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerID="eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee" exitCode=0
Jul 09 15:50:57.046907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerDied","Data":"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"}
Jul 09 15:50:57.046907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046695    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"
Jul 09 15:50:57.046907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046727    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerDied","Data":"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"}
Jul 09 15:50:57.046907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2" event={"ID":"1ad131b5-831f-4c6e-bb23-ade275d7f79f","Type":"ContainerDied","Data":"c4656f5c4bd44440c2f9736b0418b80296213dd3c62ef65a162cfefe1e7d1480"}
Jul 09 15:50:57.046907 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.046782    2578 scope.go:117] "RemoveContainer" containerID="87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"
Jul 09 15:50:57.055043 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.055024    2578 scope.go:117] "RemoveContainer" containerID="eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"
Jul 09 15:50:57.062073 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.062055    2578 scope.go:117] "RemoveContainer" containerID="3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"
Jul 09 15:50:57.069256 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.069240    2578 scope.go:117] "RemoveContainer" containerID="87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"
Jul 09 15:50:57.069513 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:57.069495    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a\": container with ID starting with 87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a not found: ID does not exist" containerID="87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"
Jul 09 15:50:57.069558 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.069522    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"} err="failed to get container status \"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a\": rpc error: code = NotFound desc = could not find container \"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a\": container with ID starting with 87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a not found: ID does not exist"
Jul 09 15:50:57.069558 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.069540    2578 scope.go:117] "RemoveContainer" containerID="eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"
Jul 09 15:50:57.069789 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:57.069765    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee\": container with ID starting with eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee not found: ID does not exist" containerID="eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"
Jul 09 15:50:57.069870 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.069802    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"} err="failed to get container status \"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee\": rpc error: code = NotFound desc = could not find container \"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee\": container with ID starting with eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee not found: ID does not exist"
Jul 09 15:50:57.069870 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.069825    2578 scope.go:117] "RemoveContainer" containerID="3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"
Jul 09 15:50:57.070053 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:50:57.070038    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074\": container with ID starting with 3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074 not found: ID does not exist" containerID="3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"
Jul 09 15:50:57.070092 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070057    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"} err="failed to get container status \"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074\": rpc error: code = NotFound desc = could not find container \"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074\": container with ID starting with 3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074 not found: ID does not exist"
Jul 09 15:50:57.070092 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070070    2578 scope.go:117] "RemoveContainer" containerID="87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"
Jul 09 15:50:57.070236 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070219    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a"} err="failed to get container status \"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a\": rpc error: code = NotFound desc = could not find container \"87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a\": container with ID starting with 87f869238dc38044f95602d76dd9bac28d6af6e25dd27e0543b023c996bcc83a not found: ID does not exist"
Jul 09 15:50:57.070281 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070236    2578 scope.go:117] "RemoveContainer" containerID="eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"
Jul 09 15:50:57.070454 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070436    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee"} err="failed to get container status \"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee\": rpc error: code = NotFound desc = could not find container \"eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee\": container with ID starting with eec070b3c81366f6cbdb17066426f7f5053d00b5c3cee6e06cf42a8275dfceee not found: ID does not exist"
Jul 09 15:50:57.070528 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070456    2578 scope.go:117] "RemoveContainer" containerID="3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"
Jul 09 15:50:57.070696 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.070678    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074"} err="failed to get container status \"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074\": rpc error: code = NotFound desc = could not find container \"3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074\": container with ID starting with 3e2a71ae6b1c7be388cce42eb4e281d523e1f2ffb7d446c922945514a95f3074 not found: ID does not exist"
Jul 09 15:50:57.081020 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls\") pod \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") "
Jul 09 15:50:57.081099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rqxs9\" (UniqueName: \"kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9\") pod \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") "
Jul 09 15:50:57.081099 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081049    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config\") pod \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") "
Jul 09 15:50:57.081209 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081103    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert\") pod \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") "
Jul 09 15:50:57.081209 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location\") pod \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\" (UID: \"1ad131b5-831f-4c6e-bb23-ade275d7f79f\") "
Jul 09 15:50:57.081464 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081437    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "1ad131b5-831f-4c6e-bb23-ade275d7f79f" (UID: "1ad131b5-831f-4c6e-bb23-ade275d7f79f"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:50:57.081464 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "1ad131b5-831f-4c6e-bb23-ade275d7f79f" (UID: "1ad131b5-831f-4c6e-bb23-ade275d7f79f"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 15:50:57.082011 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.081989    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "1ad131b5-831f-4c6e-bb23-ade275d7f79f" (UID: "1ad131b5-831f-4c6e-bb23-ade275d7f79f"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:50:57.083060 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.083042    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "1ad131b5-831f-4c6e-bb23-ade275d7f79f" (UID: "1ad131b5-831f-4c6e-bb23-ade275d7f79f"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 15:50:57.083252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.083234    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9" (OuterVolumeSpecName: "kube-api-access-rqxs9") pod "1ad131b5-831f-4c6e-bb23-ade275d7f79f" (UID: "1ad131b5-831f-4c6e-bb23-ade275d7f79f"). InnerVolumeSpecName "kube-api-access-rqxs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:50:57.182231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.182207    2578 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-cabundle-cert\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:57.182231 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.182230    2578 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kserve-provision-location\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:57.182540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.182243    2578 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/1ad131b5-831f-4c6e-bb23-ade275d7f79f-proxy-tls\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:57.182540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.182252    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqxs9\" (UniqueName: \"kubernetes.io/projected/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-api-access-rqxs9\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:57.182540 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.182259    2578 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/1ad131b5-831f-4c6e-bb23-ade275d7f79f-kube-rbac-proxy-sar-config\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:50:57.374239 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.374210    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:57.378864 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:57.378843    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-autogluon-ts-v1-predictor-89d67dbf7-2mbg2"]
Jul 09 15:50:58.104198 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:50:58.104166    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" path="/var/lib/kubelet/pods/1ad131b5-831f-4c6e-bb23-ade275d7f79f/volumes"
Jul 09 15:51:02.765875 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.765844    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8qsc5/must-gather-z2242"]
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766159    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="storage-initializer"
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766170    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="storage-initializer"
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766182    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container"
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766187    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container"
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766208    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kube-rbac-proxy"
Jul 09 15:51:02.766233 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766215    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kube-rbac-proxy"
Jul 09 15:51:02.766426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766273    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kube-rbac-proxy"
Jul 09 15:51:02.766426 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.766285    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ad131b5-831f-4c6e-bb23-ade275d7f79f" containerName="kserve-container"
Jul 09 15:51:02.770967 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.770947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.773582 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.773562    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8qsc5\"/\"openshift-service-ca.crt\""
Jul 09 15:51:02.775121 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.775097    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8qsc5\"/\"kube-root-ca.crt\""
Jul 09 15:51:02.775240 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.775149    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8qsc5\"/\"default-dockercfg-n92lp\""
Jul 09 15:51:02.784010 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.783989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8qsc5/must-gather-z2242"]
Jul 09 15:51:02.829016 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.828987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.829124 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.829054    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zc6gc\" (UniqueName: \"kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.929522 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.929495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.929633 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.929545    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zc6gc\" (UniqueName: \"kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.929865 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.929846    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:02.940479 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:02.940458    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zc6gc\" (UniqueName: \"kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc\") pod \"must-gather-z2242\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") " pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:03.098032 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:03.097954    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:03.228501 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:03.228468    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8qsc5/must-gather-z2242"]
Jul 09 15:51:03.234165 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:51:03.234124    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a700d4a_fc63_4f35_9194_1a0f8eb5c413.slice/crio-f8e1ee5b2a119984b7a6181868b91cd56b49eb313bd4565b338db9c2d6a9b678 WatchSource:0}: Error finding container f8e1ee5b2a119984b7a6181868b91cd56b49eb313bd4565b338db9c2d6a9b678: Status 404 returned error can't find the container with id f8e1ee5b2a119984b7a6181868b91cd56b49eb313bd4565b338db9c2d6a9b678
Jul 09 15:51:03.236008 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:03.235991    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 15:51:04.080271 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:04.080195    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qsc5/must-gather-z2242" event={"ID":"5a700d4a-fc63-4f35-9194-1a0f8eb5c413","Type":"ContainerStarted","Data":"f8e1ee5b2a119984b7a6181868b91cd56b49eb313bd4565b338db9c2d6a9b678"}
Jul 09 15:51:10.110563 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:10.110523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qsc5/must-gather-z2242" event={"ID":"5a700d4a-fc63-4f35-9194-1a0f8eb5c413","Type":"ContainerStarted","Data":"ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2"}
Jul 09 15:51:10.111068 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:10.110632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qsc5/must-gather-z2242" event={"ID":"5a700d4a-fc63-4f35-9194-1a0f8eb5c413","Type":"ContainerStarted","Data":"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"}
Jul 09 15:51:10.130325 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:10.130275    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8qsc5/must-gather-z2242" podStartSLOduration=1.9775372089999999 podStartE2EDuration="8.130257698s" totalImagesPullingTime="6.152720489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:51:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:51:10.1294218 +0000 UTC m=+1476.589912799" watchObservedRunningTime="2026-07-09 15:51:10.130257698 +0000 UTC m=+1476.590749041"
Jul 09 15:51:30.205537 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:30.205499    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerID="5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a" exitCode=0
Jul 09 15:51:30.205900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:30.205547    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8qsc5/must-gather-z2242" event={"ID":"5a700d4a-fc63-4f35-9194-1a0f8eb5c413","Type":"ContainerDied","Data":"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"}
Jul 09 15:51:30.205900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:30.205848    2578 scope.go:117] "RemoveContainer" containerID="5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"
Jul 09 15:51:31.173044 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.173017    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-8qsc5_must-gather-z2242_5a700d4a-fc63-4f35-9194-1a0f8eb5c413/gather/0.log"
Jul 09 15:51:31.695947 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.695911    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mlhdd/must-gather-82bxq"]
Jul 09 15:51:31.698841 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.698819    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.701241 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.701223    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mlhdd\"/\"default-dockercfg-wfnl2\""
Jul 09 15:51:31.701342 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.701272    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlhdd\"/\"openshift-service-ca.crt\""
Jul 09 15:51:31.701436 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.701418    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mlhdd\"/\"kube-root-ca.crt\""
Jul 09 15:51:31.713774 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.712871    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlhdd/must-gather-82bxq"]
Jul 09 15:51:31.805776 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.805727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-must-gather-output\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.805926 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.805880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfhv7\" (UniqueName: \"kubernetes.io/projected/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-kube-api-access-pfhv7\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.907129 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.907094    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfhv7\" (UniqueName: \"kubernetes.io/projected/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-kube-api-access-pfhv7\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.907303 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.907153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-must-gather-output\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.907463 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.907447    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-must-gather-output\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:31.917009 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:31.916986    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfhv7\" (UniqueName: \"kubernetes.io/projected/e8d5f940-2607-4e06-b7e5-b6f8f14baf33-kube-api-access-pfhv7\") pod \"must-gather-82bxq\" (UID: \"e8d5f940-2607-4e06-b7e5-b6f8f14baf33\") " pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:32.008095 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:32.008027    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlhdd/must-gather-82bxq"
Jul 09 15:51:32.136959 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:32.136936    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlhdd/must-gather-82bxq"]
Jul 09 15:51:32.139603 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:51:32.139573    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8d5f940_2607_4e06_b7e5_b6f8f14baf33.slice/crio-c044ef1bf07a32403a79d670739b846710e4ccc630eab29bab5277f1c7e5cc8c WatchSource:0}: Error finding container c044ef1bf07a32403a79d670739b846710e4ccc630eab29bab5277f1c7e5cc8c: Status 404 returned error can't find the container with id c044ef1bf07a32403a79d670739b846710e4ccc630eab29bab5277f1c7e5cc8c
Jul 09 15:51:32.214565 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:32.214532    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlhdd/must-gather-82bxq" event={"ID":"e8d5f940-2607-4e06-b7e5-b6f8f14baf33","Type":"ContainerStarted","Data":"c044ef1bf07a32403a79d670739b846710e4ccc630eab29bab5277f1c7e5cc8c"}
Jul 09 15:51:33.221774 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:33.221657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlhdd/must-gather-82bxq" event={"ID":"e8d5f940-2607-4e06-b7e5-b6f8f14baf33","Type":"ContainerStarted","Data":"21bf5c224749ac4679dfa9ca8d36f205fdef8f1a0e042c3649c0d6f2d54e7b82"}
Jul 09 15:51:33.221774 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:33.221717    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlhdd/must-gather-82bxq" event={"ID":"e8d5f940-2607-4e06-b7e5-b6f8f14baf33","Type":"ContainerStarted","Data":"bbf1b8663af7a4abc36215a18231e73be1c090698977adc66ec5a2e72ca88a8b"}
Jul 09 15:51:33.244083 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:33.242314    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mlhdd/must-gather-82bxq" podStartSLOduration=1.5110286419999999 podStartE2EDuration="2.242291884s" totalImagesPullingTime="731.263242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 15:51:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:51:33.241077125 +0000 UTC m=+1499.701568130" watchObservedRunningTime="2026-07-09 15:51:33.242291884 +0000 UTC m=+1499.702782883"
Jul 09 15:51:34.077303 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:34.077278    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:51:34.077938 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:34.077914    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:51:34.470603 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:34.470519    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h42wx_6660f188-abdc-44da-8487-cadab2eb96e7/global-pull-secret-syncer/0.log"
Jul 09 15:51:34.579671 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:34.579638    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vww99_04ca77ba-3e4d-4bb5-8522-ee755d697ab9/konnectivity-agent/0.log"
Jul 09 15:51:34.602900 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:34.602873    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-248.ec2.internal_9a6b91f137a83594b4e25554453bfc87/haproxy/0.log"
Jul 09 15:51:36.541678 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.541633    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-8qsc5/must-gather-z2242"]
Jul 09 15:51:36.542169 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.541977    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-must-gather-8qsc5/must-gather-z2242" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="copy" containerID="cri-o://ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2" gracePeriod=2
Jul 09 15:51:36.544271 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:36.544226    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z2242\" is forbidden: User \"system:node:ip-10-0-132-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-8qsc5\": no relationship found between node 'ip-10-0-132-248.ec2.internal' and this object" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:36.547880 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.546918    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-8qsc5/must-gather-z2242"]
Jul 09 15:51:36.926730 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.926653    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-8qsc5_must-gather-z2242_5a700d4a-fc63-4f35-9194-1a0f8eb5c413/copy/0.log"
Jul 09 15:51:36.930530 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.930261    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:36.934212 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:36.934174    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z2242\" is forbidden: User \"system:node:ip-10-0-132-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-8qsc5\": no relationship found between node 'ip-10-0-132-248.ec2.internal' and this object" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:36.970241 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.970210    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zc6gc\" (UniqueName: \"kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc\") pod \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") "
Jul 09 15:51:36.970553 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.970535    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output\") pod \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\" (UID: \"5a700d4a-fc63-4f35-9194-1a0f8eb5c413\") "
Jul 09 15:51:36.971891 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.971859    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "5a700d4a-fc63-4f35-9194-1a0f8eb5c413" (UID: "5a700d4a-fc63-4f35-9194-1a0f8eb5c413"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 15:51:36.984963 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:36.984936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc" (OuterVolumeSpecName: "kube-api-access-zc6gc") pod "5a700d4a-fc63-4f35-9194-1a0f8eb5c413" (UID: "5a700d4a-fc63-4f35-9194-1a0f8eb5c413"). InnerVolumeSpecName "kube-api-access-zc6gc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 15:51:37.071709 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.071659    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zc6gc\" (UniqueName: \"kubernetes.io/projected/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-kube-api-access-zc6gc\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:51:37.071709 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.071709    2578 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5a700d4a-fc63-4f35-9194-1a0f8eb5c413-must-gather-output\") on node \"ip-10-0-132-248.ec2.internal\" DevicePath \"\""
Jul 09 15:51:37.247568 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.247487    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-8qsc5_must-gather-z2242_5a700d4a-fc63-4f35-9194-1a0f8eb5c413/copy/0.log"
Jul 09 15:51:37.247979 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.247945    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerID="ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2" exitCode=143
Jul 09 15:51:37.248110 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.248021    2578 scope.go:117] "RemoveContainer" containerID="ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2"
Jul 09 15:51:37.248196 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.248175    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:37.258354 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:37.258315    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z2242\" is forbidden: User \"system:node:ip-10-0-132-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-8qsc5\": no relationship found between node 'ip-10-0-132-248.ec2.internal' and this object" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:37.273059 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:37.273019    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"must-gather-z2242\" is forbidden: User \"system:node:ip-10-0-132-248.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-8qsc5\": no relationship found between node 'ip-10-0-132-248.ec2.internal' and this object" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" pod="openshift-must-gather-8qsc5/must-gather-z2242"
Jul 09 15:51:37.298805 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.298227    2578 scope.go:117] "RemoveContainer" containerID="5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"
Jul 09 15:51:37.327972 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.327646    2578 scope.go:117] "RemoveContainer" containerID="ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2"
Jul 09 15:51:37.328107 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:37.328073    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2\": container with ID starting with ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2 not found: ID does not exist" containerID="ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2"
Jul 09 15:51:37.328187 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.328121    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2"} err="failed to get container status \"ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2\": rpc error: code = NotFound desc = could not find container \"ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2\": container with ID starting with ed4f2f35b4de97f10355dfe5139753cfe5944e551f11b9867361203e0ef25cd2 not found: ID does not exist"
Jul 09 15:51:37.328187 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.328148    2578 scope.go:117] "RemoveContainer" containerID="5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"
Jul 09 15:51:37.328458 ip-10-0-132-248 kubenswrapper[2578]: E0709 15:51:37.328424    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a\": container with ID starting with 5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a not found: ID does not exist" containerID="5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"
Jul 09 15:51:37.328556 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:37.328456    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a"} err="failed to get container status \"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a\": rpc error: code = NotFound desc = could not find container \"5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a\": container with ID starting with 5436065c1ce21bf77df74b10129bbf9c898edae615ee65c46aaf81836fbb195a not found: ID does not exist"
Jul 09 15:51:38.107740 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.107701    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" path="/var/lib/kubelet/pods/5a700d4a-fc63-4f35-9194-1a0f8eb5c413/volumes"
Jul 09 15:51:38.587240 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.587208    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/alertmanager/0.log"
Jul 09 15:51:38.622460 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.622431    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/config-reloader/0.log"
Jul 09 15:51:38.649715 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.649599    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/kube-rbac-proxy-web/0.log"
Jul 09 15:51:38.683634 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.683599    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/kube-rbac-proxy/0.log"
Jul 09 15:51:38.711048 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.710941    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/kube-rbac-proxy-metric/0.log"
Jul 09 15:51:38.746646 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.746615    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/prom-label-proxy/0.log"
Jul 09 15:51:38.777345 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.777308    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_1006d858-b8d6-40f3-b8b3-adfde31d6e62/init-config-reloader/0.log"
Jul 09 15:51:38.850088 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.849939    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cm86b_fc763105-2ba6-45fa-9585-76f09bd4d94d/kube-state-metrics/0.log"
Jul 09 15:51:38.883653 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.883610    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cm86b_fc763105-2ba6-45fa-9585-76f09bd4d94d/kube-rbac-proxy-main/0.log"
Jul 09 15:51:38.914868 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.914769    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-cm86b_fc763105-2ba6-45fa-9585-76f09bd4d94d/kube-rbac-proxy-self/0.log"
Jul 09 15:51:38.953964 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.953865    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6455bc7799-wh8xf_f13c58b3-a475-4cc1-8f9c-65df8b952cda/metrics-server/0.log"
Jul 09 15:51:38.981731 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:38.981645    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-s5dfq_7d058837-d814-40ea-9ef6-82394d96ea51/monitoring-plugin/0.log"
Jul 09 15:51:39.021133 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.021104    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mf8sj_017c6d42-a371-44b4-835f-6337ab0bb98a/node-exporter/0.log"
Jul 09 15:51:39.043640 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.043610    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mf8sj_017c6d42-a371-44b4-835f-6337ab0bb98a/kube-rbac-proxy/0.log"
Jul 09 15:51:39.069693 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.069660    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mf8sj_017c6d42-a371-44b4-835f-6337ab0bb98a/init-textfile/0.log"
Jul 09 15:51:39.271699 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.271669    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-r265z_a268dbc2-70ec-4b03-b96f-6caf5dbb9540/kube-rbac-proxy-main/0.log"
Jul 09 15:51:39.296089 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.296053    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-r265z_a268dbc2-70ec-4b03-b96f-6caf5dbb9540/kube-rbac-proxy-self/0.log"
Jul 09 15:51:39.321921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.321889    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-r265z_a268dbc2-70ec-4b03-b96f-6caf5dbb9540/openshift-state-metrics/0.log"
Jul 09 15:51:39.359739 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.359710    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/prometheus/0.log"
Jul 09 15:51:39.383977 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.383944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/config-reloader/0.log"
Jul 09 15:51:39.413612 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.413583    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/thanos-sidecar/0.log"
Jul 09 15:51:39.443895 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.443869    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/kube-rbac-proxy-web/0.log"
Jul 09 15:51:39.474129 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.474058    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/kube-rbac-proxy/0.log"
Jul 09 15:51:39.498387 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.498361    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/kube-rbac-proxy-thanos/0.log"
Jul 09 15:51:39.522036 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.522012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_9a8e7f05-26f5-4e04-9252-7133db41dbbf/init-config-reloader/0.log"
Jul 09 15:51:39.553136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.553105    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rkfw9_dda39670-c931-41e8-a6b8-e60ad725f98f/prometheus-operator/0.log"
Jul 09 15:51:39.577172 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.577136    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-rkfw9_dda39670-c931-41e8-a6b8-e60ad725f98f/kube-rbac-proxy/0.log"
Jul 09 15:51:39.653085 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.653059    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485bd4bf-p6sfb_07adf231-3ea6-485b-8bea-5142b1dfeb98/telemeter-client/0.log"
Jul 09 15:51:39.681186 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.681152    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485bd4bf-p6sfb_07adf231-3ea6-485b-8bea-5142b1dfeb98/reload/0.log"
Jul 09 15:51:39.708117 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.708093    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8485bd4bf-p6sfb_07adf231-3ea6-485b-8bea-5142b1dfeb98/kube-rbac-proxy/0.log"
Jul 09 15:51:39.745941 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.745861    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/thanos-query/0.log"
Jul 09 15:51:39.773006 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.772981    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/kube-rbac-proxy-web/0.log"
Jul 09 15:51:39.800252 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.800214    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/kube-rbac-proxy/0.log"
Jul 09 15:51:39.828783 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.828731    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/prom-label-proxy/0.log"
Jul 09 15:51:39.856195 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.856149    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/kube-rbac-proxy-rules/0.log"
Jul 09 15:51:39.884124 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:39.884093    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-684ff48bb7-l946t_f3abd3de-08f5-42d9-a560-19690bbccd3c/kube-rbac-proxy-metrics/0.log"
Jul 09 15:51:42.210833 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.210777    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bb7d6d4c9-8g7r2_8dd56f4a-e51e-493a-a02c-8c62833fed66/console/0.log"
Jul 09 15:51:42.427447 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.427410    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"]
Jul 09 15:51:42.428025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.427990    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="gather"
Jul 09 15:51:42.428025 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.428020    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="gather"
Jul 09 15:51:42.428206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.428036    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="copy"
Jul 09 15:51:42.428206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.428045    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="copy"
Jul 09 15:51:42.428206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.428152    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="copy"
Jul 09 15:51:42.428206 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.428162    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a700d4a-fc63-4f35-9194-1a0f8eb5c413" containerName="gather"
Jul 09 15:51:42.431822 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.431800    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.439959 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.439932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"]
Jul 09 15:51:42.537164 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.537125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-podres\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.537380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.537214    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skv7z\" (UniqueName: \"kubernetes.io/projected/87595926-872e-4136-98e1-a908bc9f3c8f-kube-api-access-skv7z\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.537380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.537290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-sys\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.537380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.537322    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-proc\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.537380 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.537346    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-lib-modules\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.637945 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.637907    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skv7z\" (UniqueName: \"kubernetes.io/projected/87595926-872e-4136-98e1-a908bc9f3c8f-kube-api-access-skv7z\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-sys\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-proc\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-lib-modules\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638317 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-proc\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638317 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-podres\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638317 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638223    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-lib-modules\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638317 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638283    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-sys\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.638523 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.638337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87595926-872e-4136-98e1-a908bc9f3c8f-podres\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.647654 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.647623    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skv7z\" (UniqueName: \"kubernetes.io/projected/87595926-872e-4136-98e1-a908bc9f3c8f-kube-api-access-skv7z\") pod \"perf-node-gather-daemonset-j92ln\" (UID: \"87595926-872e-4136-98e1-a908bc9f3c8f\") " pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.747200 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.747165    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:42.901215 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:42.900726    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"]
Jul 09 15:51:42.906189 ip-10-0-132-248 kubenswrapper[2578]: W0709 15:51:42.905336    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod87595926_872e_4136_98e1_a908bc9f3c8f.slice/crio-879d2b82a9f43b0553a1bb1f3227246e1f02725de18aca80de9a737159f046e1 WatchSource:0}: Error finding container 879d2b82a9f43b0553a1bb1f3227246e1f02725de18aca80de9a737159f046e1: Status 404 returned error can't find the container with id 879d2b82a9f43b0553a1bb1f3227246e1f02725de18aca80de9a737159f046e1
Jul 09 15:51:43.287494 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.287461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln" event={"ID":"87595926-872e-4136-98e1-a908bc9f3c8f","Type":"ContainerStarted","Data":"753d919671dadcf2c8f1a1d2ad77b5a53f025c57b927dddddeb7a35d05b2548a"}
Jul 09 15:51:43.287921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.287512    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln" event={"ID":"87595926-872e-4136-98e1-a908bc9f3c8f","Type":"ContainerStarted","Data":"879d2b82a9f43b0553a1bb1f3227246e1f02725de18aca80de9a737159f046e1"}
Jul 09 15:51:43.287921 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.287562    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:43.305393 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.305334    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln" podStartSLOduration=1.305314287 podStartE2EDuration="1.305314287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 15:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 15:51:43.304109456 +0000 UTC m=+1509.764600454" watchObservedRunningTime="2026-07-09 15:51:43.305314287 +0000 UTC m=+1509.765805285"
Jul 09 15:51:43.843136 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.843112    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thqtm_de21f5ac-bf48-45f1-b93c-696a5e9b8dec/dns/0.log"
Jul 09 15:51:43.867014 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.866977    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thqtm_de21f5ac-bf48-45f1-b93c-696a5e9b8dec/kube-rbac-proxy/0.log"
Jul 09 15:51:43.971118 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:43.971081    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jj6lk_66df486a-05c7-4e72-b2e0-bce404d9ccac/dns-node-resolver/0.log"
Jul 09 15:51:44.489899 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:44.489871    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hj47z_6c762516-924f-47e5-9db9-2d92680a5c6a/node-ca/0.log"
Jul 09 15:51:45.910115 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:45.910083    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x697l_2453b98e-735e-47cf-927b-d3f2ee1661d2/serve-healthcheck-canary/0.log"
Jul 09 15:51:46.474171 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:46.474143    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r7dgw_2fb8f942-6acc-436c-aed4-baad18db12ae/kube-rbac-proxy/0.log"
Jul 09 15:51:46.499063 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:46.499040    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r7dgw_2fb8f942-6acc-436c-aed4-baad18db12ae/exporter/0.log"
Jul 09 15:51:46.523247 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:46.523215    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r7dgw_2fb8f942-6acc-436c-aed4-baad18db12ae/extractor/0.log"
Jul 09 15:51:48.830532 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.830498    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-9zgrg_60cae34a-5134-4cef-8d4a-397c2675b335/s3-init/0.log"
Jul 09 15:51:48.847794 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.847745    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-9zgrg_60cae34a-5134-4cef-8d4a-397c2675b335/download-hf-model/0.log"
Jul 09 15:51:48.878574 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.878548    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-dnd58_45d5c8e7-476e-4254-a986-aeded13a8180/s3-tls-init-custom/0.log"
Jul 09 15:51:48.896307 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.896288    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-custom-dnd58_45d5c8e7-476e-4254-a986-aeded13a8180/download-hf-model/0.log"
Jul 09 15:51:48.926196 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.926173    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-lp2hx_026fff70-99da-49de-827d-8ff28ee7446c/s3-tls-init-serving/0.log"
Jul 09 15:51:48.943369 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.943349    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-tls-init-serving-lp2hx_026fff70-99da-49de-827d-8ff28ee7446c/download-hf-model/0.log"
Jul 09 15:51:48.996291 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:48.996272    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-tls-custom-8b44d767d-2h4c8_fa15aff0-6561-4d23-9940-0e961ec04b80/seaweedfs-tls-custom/0.log"
Jul 09 15:51:49.307809 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:49.307781    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mlhdd/perf-node-gather-daemonset-j92ln"
Jul 09 15:51:54.827373 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.827299    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-648lm_950abca6-2db7-4906-9089-dece19bf4781/kube-multus/0.log"
Jul 09 15:51:54.877697 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.877669    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/kube-multus-additional-cni-plugins/0.log"
Jul 09 15:51:54.899926 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.899900    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/egress-router-binary-copy/0.log"
Jul 09 15:51:54.924410 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.924390    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/cni-plugins/0.log"
Jul 09 15:51:54.947203 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.947183    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/bond-cni-plugin/0.log"
Jul 09 15:51:54.986461 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:54.986446    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/routeoverride-cni/0.log"
Jul 09 15:51:55.012876 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:55.012854    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/whereabouts-cni-bincopy/0.log"
Jul 09 15:51:55.037075 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:55.037057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-r9sjn_66e56408-1d75-4c3b-ac75-bf0c62ee4f07/whereabouts-cni/0.log"
Jul 09 15:51:55.485804 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:55.485779    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9ktpc_3d446ecc-426d-465e-96b6-d42360901d7f/network-metrics-daemon/0.log"
Jul 09 15:51:55.510214 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:55.510189    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-9ktpc_3d446ecc-426d-465e-96b6-d42360901d7f/kube-rbac-proxy/0.log"
Jul 09 15:51:56.589799 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.589769    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-controller/0.log"
Jul 09 15:51:56.610571 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.610548    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/1.log"
Jul 09 15:51:56.610695 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.610631    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovn-acl-logging/0.log"
Jul 09 15:51:56.631478 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.631457    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/kube-rbac-proxy-node/0.log"
Jul 09 15:51:56.654547 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.654527    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 15:51:56.678416 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.678394    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/northd/0.log"
Jul 09 15:51:56.718518 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.718491    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/nbdb/0.log"
Jul 09 15:51:56.742769 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.742728    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/sbdb/0.log"
Jul 09 15:51:56.790489 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:56.790464    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-qprmf_de5803a7-2750-43cb-89bd-4fea94ea4898/ovnkube-controller/0.log"
Jul 09 15:51:58.154845 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:58.154817    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vg5h6_734914ef-7a03-4867-b5f1-a6cbf0e26d78/network-check-target-container/0.log"
Jul 09 15:51:59.142167 ip-10-0-132-248 kubenswrapper[2578]: I0709 15:51:59.142097    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-mqvjf_a16510d5-86e7-40f6-aa55-b5ed60ec4fe7/iptables-alerter/0.log"