Jul 18 01:48:42.375419 ip-10-0-129-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 18 01:48:42.375433 ip-10-0-129-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 18 01:48:42.375443 ip-10-0-129-141 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 18 01:48:42.375760 ip-10-0-129-141 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 18 01:48:52.624417 ip-10-0-129-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 18 01:48:52.624440 ip-10-0-129-141 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 13b733347fc54bb5818a50e62309fb3a --
Jul 18 01:50:30.573897 ip-10-0-129-141 systemd[1]: Starting Kubernetes Kubelet...
Jul 18 01:50:30.903233 ip-10-0-129-141 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:50:30.903233 ip-10-0-129-141 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 18 01:50:30.903233 ip-10-0-129-141 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:50:30.903233 ip-10-0-129-141 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 18 01:50:30.903233 ip-10-0-129-141 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 18 01:50:30.905388 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.905238    2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 18 01:50:30.908797 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908775    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.908797 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908798    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908802    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908805    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908812    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908815    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908818    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908821    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908823    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908826    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908828    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908831    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908834    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908836    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908838    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908841    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908844    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908846    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908849    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908851    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908853    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.908877 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908855    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908858    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908861    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908865    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908868    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908872    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908874    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908877    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908880    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908883    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908886    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908888    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908891    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908894    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908896    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908898    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908902    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908904    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908907    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.909358 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908909    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908913    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908915    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908918    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908920    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908924    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908927    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908930    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908933    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908935    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908938    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908941    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908944    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908946    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908949    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908952    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908954    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908956    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908960    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.909945 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908962    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908965    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908967    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908970    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908972    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908975    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908977    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908979    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908982    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908984    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908987    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908990    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908992    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908994    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908997    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.908999    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909001    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909004    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909006    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909009    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.910782 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909011    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909013    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909015    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909018    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909021    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909023    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909026    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909028    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909031    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909034    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909036    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909042    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909045    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909048    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909051    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909054    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909056    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909059    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909062    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.909065    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.911275 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911091    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911107    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911112    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911115    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911118    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911121    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911124    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911126    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911140    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911143    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911146    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911148    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911151    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911154    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911156    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911159    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911161    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911164    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911166    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911169    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.911770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911172    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911174    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911176    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911179    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911181    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911184    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911186    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911188    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911191    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911193    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911199    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911202    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911205    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911208    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911211    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911213    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911216    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911219    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911221    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.912261 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911224    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911227    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911230    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911232    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911234    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911237    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911239    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911242    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911244    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911247    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911249    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911254    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911257    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911260    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911263    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911265    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911268    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911270    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911273    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.912741 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911275    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911279    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911281    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911284    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911287    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911289    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911294    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911299    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911303    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911309    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911314    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911317    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911320    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911323    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911326    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911328    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911331    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911333    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911336    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.913211 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911339    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911341    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911343    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911346    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911348    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911351    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911354    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911357    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911359    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911361    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911364    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911366    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911368    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911371    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911374    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911377    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911379    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911381    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911384    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911387    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911390    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.913689 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911392    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911891    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911898    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911901    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911904    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911907    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911910    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911913    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911915    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911918    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911920    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911923    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911926    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911929    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911931    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911934    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911937    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911940    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911943    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911945    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.914191 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911948    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911952    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911954    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911956    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911959    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911963    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911966    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911969    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911972    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911975    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911977    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911980    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911983    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911987    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911991    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911993    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911996    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.911998    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912001    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.914710 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912003    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912006    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912008    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912010    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912013    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912015    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912018    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912021    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912023    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912026    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912028    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912031    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912033    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912036    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912038    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912041    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912043    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912046    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912048    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.915182 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912051    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912054    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912056    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912059    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912061    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912064    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912068    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912071    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912074    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912077    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912080    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912082    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912085    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912087    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912090    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912092    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912095    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912097    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912100    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912102    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.915697 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912104    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912107    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912109    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912112    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912114    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912118    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912120    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912123    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912125    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912127    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912130    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912133    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912135    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912137    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912140    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912142    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912145    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912147    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912149    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912151    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.916197 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912154    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912156    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912248    2560 flags.go:64] FLAG: --address="0.0.0.0"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912258    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912268    2560 flags.go:64] FLAG: --anonymous-auth="true"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912276    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912280    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912283    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912287    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912292    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912295    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912298    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912301    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912305    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912307    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912310    2560 flags.go:64] FLAG: --cgroup-root=""
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912313    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912317    2560 flags.go:64] FLAG: --client-ca-file=""
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912322    2560 flags.go:64] FLAG: --cloud-provider="external"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912325    2560 flags.go:64] FLAG: --cluster-dns="[]"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912334    2560 flags.go:64] FLAG: --cluster-domain=""
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912337    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912340    2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912344    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 18 01:50:30.916695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912347    2560 flags.go:64] FLAG: --container-log-max-files="5"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912351    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912354    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912357    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912360    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912363    2560 flags.go:64] FLAG: --contention-profiling="false"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912366    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912369    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912372    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912375    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912379    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912382    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912385    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912387    2560 flags.go:64] FLAG: --enable-load-reader="false"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912390    2560 flags.go:64] FLAG: --enable-server="true"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912393    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912397    2560 flags.go:64] FLAG: --event-burst="100"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912400    2560 flags.go:64] FLAG: --event-qps="50"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912403    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912406    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912409    2560 flags.go:64] FLAG: --eviction-hard=""
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912413    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912416    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912419    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912421    2560 flags.go:64] FLAG: --eviction-soft=""
Jul 18 01:50:30.917271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912424    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912429    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912432    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912436    2560 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912439    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912442    2560 flags.go:64] FLAG: --fail-swap-on="true"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912445    2560 flags.go:64] FLAG: --feature-gates=""
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912449    2560 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912452    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912455    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912458    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912461    2560 flags.go:64] FLAG: --healthz-port="10248"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912464    2560 flags.go:64] FLAG: --help="false"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912466    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-129-141.ec2.internal"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912469    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912472    2560 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912475    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912478    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912482    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912485    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912487    2560 flags.go:64] FLAG: --image-service-endpoint=""
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912490    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912493    2560 flags.go:64] FLAG: --kube-api-burst="100"
Jul 18 01:50:30.917887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912496    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912498    2560 flags.go:64] FLAG: --kube-api-qps="50"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912501    2560 flags.go:64] FLAG: --kube-reserved=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912504    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912507    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912510    2560 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912512    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912528    2560 flags.go:64] FLAG: --lock-file=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912531    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912534    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912538    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912544    2560 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912549    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912552    2560 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912555    2560 flags.go:64] FLAG: --logging-format="text"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912557    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912561    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912564    2560 flags.go:64] FLAG: --manifest-url=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912567    2560 flags.go:64] FLAG: --manifest-url-header=""
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912571    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912574    2560 flags.go:64] FLAG: --max-open-files="1000000"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912578    2560 flags.go:64] FLAG: --max-pods="110"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912581    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912585    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912587    2560 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 18 01:50:30.918469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912590    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912593    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912596    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912598    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912607    2560 flags.go:64] FLAG: --node-status-max-images="50"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912610    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912612    2560 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912615    2560 flags.go:64] FLAG: --pod-cidr=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912618    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912624    2560 flags.go:64] FLAG: --pod-manifest-path=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912627    2560 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912630    2560 flags.go:64] FLAG: --pods-per-core="0"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912633    2560 flags.go:64] FLAG: --port="10250"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912636    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912639    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ac0f188f94f826a6"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912642    2560 flags.go:64] FLAG: --qos-reserved=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912645    2560 flags.go:64] FLAG: --read-only-port="10255"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912648    2560 flags.go:64] FLAG: --register-node="true"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912652    2560 flags.go:64] FLAG: --register-with-taints=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912657    2560 flags.go:64] FLAG: --registry-burst="10"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912661    2560 flags.go:64] FLAG: --registry-qps="5"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912663    2560 flags.go:64] FLAG: --reserved-cpus=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912666    2560 flags.go:64] FLAG: --reserved-memory=""
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912670    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912673    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 18 01:50:30.919153 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912676    2560 flags.go:64] FLAG: --rotate-certificates="false"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912679    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912682    2560 flags.go:64] FLAG: --runonce="false"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912685    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912688    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912692    2560 flags.go:64] FLAG: --seccomp-default="false"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912694    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912697    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912700    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912703    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912706    2560 flags.go:64] FLAG: --storage-driver-password="root"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912709    2560 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912712    2560 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912714    2560 flags.go:64] FLAG: --storage-driver-user="root"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912718    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912720    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912724    2560 flags.go:64] FLAG: --system-cgroups=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912726    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912732    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912735    2560 flags.go:64] FLAG: --tls-cert-file=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912737    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912743    2560 flags.go:64] FLAG: --tls-min-version=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912746    2560 flags.go:64] FLAG: --tls-private-key-file=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912748    2560 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912751    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 18 01:50:30.919790 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912755    2560 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912760    2560 flags.go:64] FLAG: --v="2"
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912764    2560 flags.go:64] FLAG: --version="false"
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912769    2560 flags.go:64] FLAG: --vmodule=""
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912772    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.912776    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912903    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912907    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912910    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912912    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912915    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912917    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912920    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912923    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912926    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912930    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912933    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912936    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912939    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912942    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912944    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.920403 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912947    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912949    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912952    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912955    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912957    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912960    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912963    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912965    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912967    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912970    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912972    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912976    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912980    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912982    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912985    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912989    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912992    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.912996    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913000    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.920930 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913003    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913005    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913008    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913011    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913013    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913015    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913018    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913020    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913023    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913025    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913027    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913029    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913032    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913035    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913037    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913039    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913042    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913044    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913046    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913049    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.921404 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913051    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913053    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913056    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913058    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913063    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913067    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913069    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913072    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913074    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913077    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913080    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913083    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913085    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913088    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913090    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913093    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913095    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913097    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913100    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913102    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.921914 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913105    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913107    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913109    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913112    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913114    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913116    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913119    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913121    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913124    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913126    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913128    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913131    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913133    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913135    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913138    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913140    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913145    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913149    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913152    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.922383 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913154    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913157    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913160    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913163    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913166    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.913168    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.913651    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.920677    2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.920699    2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.920720    2560 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.920727    2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920789    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920794    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920798    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920800    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.922875 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920803    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920807    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920810    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920813    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920815    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920818    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920821    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920823    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920827    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920830    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920832    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920835    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920837    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920842    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920846    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920849    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920851    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920855    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920859    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.923251 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920863    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920865    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920869    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920871    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920874    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920877    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920882    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920885    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920888    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920891    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920893    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920896    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920899    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920901    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920904    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920906    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920909    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920911    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920913    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920916    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.923856 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920918    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920921    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920923    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920926    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920928    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920931    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920934    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920936    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920939    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920942    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920944    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920947    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920950    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920953    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920955    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920957    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920960    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920962    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920965    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920968    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.924340 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920970    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920973    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920975    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920978    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920980    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920982    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920984    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920987    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920989    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920991    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920994    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920996    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.920998    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921001    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921003    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921006    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921008    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921011    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921013    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921015    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.924834 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921018    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921020    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921022    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921025    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921029    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921032    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921035    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921037    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921040    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921042    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921045    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921048    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921051    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921054    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921056    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.925315 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921059    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.921064    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921183    2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921187    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921190    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921194    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921197    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921200    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921203    2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921206    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921208    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921211    2560 feature_gate.go:328] unrecognized feature gate: Example
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921214    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921217    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921220    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 18 01:50:30.925700 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921223    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921225    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921228    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921231    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921233    2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921236    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921240    2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921243    2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921246    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921249    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921252    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921254    2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921257    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921260    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921263    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921266    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921268    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921270    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921273    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 18 01:50:30.926069 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921275    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921277    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921280    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921282    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921284    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921287    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921289    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921291    2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921293    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921296    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921299    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921303    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921305    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921308    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921311    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921314    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921317    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921320    2560 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921322    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 18 01:50:30.926542 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921325    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921327    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921330    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921332    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921335    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921337    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921339    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921342    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921344    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921347    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921350    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921352    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921356    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921360    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921363    2560 feature_gate.go:328] unrecognized feature gate: Example2
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921365    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921368    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921370    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921372    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 18 01:50:30.927012 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921375    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921377    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921379    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921382    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921384    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921387    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921389    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921392    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921394    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921397    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921400    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921402    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921405    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921407    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921410    2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921413    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921415    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921418    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921421    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921423    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 18 01:50:30.927508 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921427    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921429    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921431    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921434    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921437    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921440    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921442    2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921445    2560 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:30.921447    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.921452    2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.921626    2560 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.926198    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.927460    2560 server.go:1019] "Starting client certificate rotation"
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.927568    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:50:30.928085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.927617    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 18 01:50:30.929597 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.929581    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:50:30.931425 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.931411    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 18 01:50:30.939213 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.939189    2560 log.go:25] "Validated CRI v1 runtime API"
Jul 18 01:50:30.941241 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.941223    2560 log.go:25] "Validated CRI v1 image API"
Jul 18 01:50:30.942041 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.942019    2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 18 01:50:30.944319 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.944298    2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 993988d0-bd16-46fb-9eb0-ac3cc6543cb5:/dev/nvme0n1p3 bf5c8e72-14e3-41ab-bd10-c5f41154c981:/dev/nvme0n1p4]
Jul 18 01:50:30.944390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.944319    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 18 01:50:30.949489 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.949361    2560 manager.go:216] Machine: {Timestamp:2026-07-18 01:50:30.948071537 +0000 UTC m=+0.270451695 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100751 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec257f4ea64adff18b25e9d736ac8e39 SystemUUID:ec257f4e-a64a-dff1-8b25-e9d736ac8e39 BootID:13b73334-7fc5-4bb5-818a-50e62309fb3a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d8:6f:35:a1:89 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d8:6f:35:a1:89 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:4a:2b:eb:24:e9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 18 01:50:30.949489 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.949475    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 18 01:50:30.949638 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.949589    2560 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 18 01:50:30.952429 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.952395    2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 18 01:50:30.952630 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.952433    2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 18 01:50:30.953105 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.953092    2560 topology_manager.go:138] "Creating topology manager with none policy"
Jul 18 01:50:30.953166 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.953111    2560 container_manager_linux.go:309] "Creating device plugin manager"
Jul 18 01:50:30.953166 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.953122    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:50:30.953698 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.953685    2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 18 01:50:30.954778 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.954765    2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 18 01:50:30.955533 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.955500    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:50:30.958692 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958677    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:50:30.958842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958830    2560 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 18 01:50:30.958909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958870    2560 kubelet.go:486] "Attempting to sync node with API server"
Jul 18 01:50:30.958909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958886    2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 18 01:50:30.958909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958901    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 18 01:50:30.959031 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958916    2560 kubelet.go:398] "Adding apiserver pod source"
Jul 18 01:50:30.959031 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.958932    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 18 01:50:30.959377 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.959358    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:50:30.959456 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.959382    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 18 01:50:30.961271 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.961251    2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 18 01:50:30.962644 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.962624    2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 18 01:50:30.962693 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.962661    2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 18 01:50:30.963302 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963291    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 18 01:50:30.963341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963309    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 18 01:50:30.963341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963315    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 18 01:50:30.963341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963321    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 18 01:50:30.963341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963331    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 18 01:50:30.963341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963337    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963345    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963352    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963359    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963364    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963378    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 18 01:50:30.963476 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.963384    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 18 01:50:30.964611 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.964597    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 18 01:50:30.964611 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.964612    2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 18 01:50:30.968530 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.968496    2560 server.go:1268] "Started kubelet"
Jul 18 01:50:30.968706 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.968630    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 18 01:50:30.969361 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.969109    2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 18 01:50:30.972223 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.972191    2560 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 18 01:50:30.972440 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.972404    2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 18 01:50:30.972771 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.972748    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 18 01:50:30.972891 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.972816    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 18 01:50:30.973381 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.973344    2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 18 01:50:30.973395 ip-10-0-129-141 systemd[1]: Started Kubernetes Kubelet.
Jul 18 01:50:30.974465 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.974449    2560 server.go:310] "Adding debug handlers to kubelet server"
Jul 18 01:50:30.978723 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.978691    2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 18 01:50:30.980146 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.979259    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-141.ec2.internal.18c33f423598f810  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-141.ec2.internal,UID:ip-10-0-129-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-141.ec2.internal,},FirstTimestamp:2026-07-18 01:50:30.968457232 +0000 UTC m=+0.290837389,LastTimestamp:2026-07-18 01:50:30.968457232 +0000 UTC m=+0.290837389,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-141.ec2.internal,}"
Jul 18 01:50:30.980382 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.980367    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 18 01:50:30.980437 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.980400    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 18 01:50:30.980489 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.980468    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:30.981407 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981380    2560 factory.go:55] Registering systemd factory
Jul 18 01:50:30.981538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981426    2560 factory.go:223] Registration of the systemd container factory successfully
Jul 18 01:50:30.981727 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981710    2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 18 01:50:30.981796 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981712    2560 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 18 01:50:30.981796 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981742    2560 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 18 01:50:30.981796 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981758    2560 factory.go:153] Registering CRI-O factory
Jul 18 01:50:30.981796 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981773    2560 factory.go:223] Registration of the crio container factory successfully
Jul 18 01:50:30.981953 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981818    2560 reconstruct.go:97] "Volume reconstruction finished"
Jul 18 01:50:30.981953 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981831    2560 reconciler.go:29] "Reconciler: start to sync state"
Jul 18 01:50:30.981953 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981820    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 18 01:50:30.981953 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981862    2560 factory.go:103] Registering Raw factory
Jul 18 01:50:30.981953 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.981876    2560 manager.go:1196] Started watching for new ooms in manager
Jul 18 01:50:30.982345 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.982333    2560 manager.go:349] Starting recovery of all containers
Jul 18 01:50:30.987092 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.987062    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 18 01:50:30.987265 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.987224    2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 18 01:50:30.994224 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.994196    2560 manager.go:354] Recovery completed
Jul 18 01:50:30.995752 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:30.995732    2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 18 01:50:30.999058 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:30.999043    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.001513 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.001479    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.001648 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.001563    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.001648 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.001576    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.002152 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002135    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 18 01:50:31.002152 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002148    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 18 01:50:31.002152 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002150    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.002312 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002171    2560 state_mem.go:36] "Initialized new in-memory state store"
Jul 18 01:50:31.002432 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002368    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4s4gj"
Jul 18 01:50:31.002988 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.002969    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.003057 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.003002    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.003057 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.003017    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.003505 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.003419    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-141.ec2.internal.18c33f423791418c  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-141.ec2.internal,UID:ip-10-0-129-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-141.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-141.ec2.internal,},FirstTimestamp:2026-07-18 01:50:31.001506188 +0000 UTC m=+0.323886348,LastTimestamp:2026-07-18 01:50:31.001506188 +0000 UTC m=+0.323886348,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-141.ec2.internal,}"
Jul 18 01:50:31.004193 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.004179    2560 policy_none.go:49] "None policy: Start"
Jul 18 01:50:31.004241 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.004197    2560 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 18 01:50:31.004241 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.004208    2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 18 01:50:31.006121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.006108    2560 policy_none.go:47] "Start"
Jul 18 01:50:31.012179 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.011942    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-4s4gj"
Jul 18 01:50:31.013385 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.013267    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-141.ec2.internal.18c33f423792388e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-141.ec2.internal,UID:ip-10-0-129-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-141.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-141.ec2.internal,},FirstTimestamp:2026-07-18 01:50:31.001569422 +0000 UTC m=+0.323949578,LastTimestamp:2026-07-18 01:50:31.001569422 +0000 UTC m=+0.323949578,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-141.ec2.internal,}"
Jul 18 01:50:31.015351 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.015280    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-141.ec2.internal.18c33f4237926169  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-141.ec2.internal,UID:ip-10-0-129-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-141.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-141.ec2.internal,},FirstTimestamp:2026-07-18 01:50:31.001579881 +0000 UTC m=+0.323960038,LastTimestamp:2026-07-18 01:50:31.001579881 +0000 UTC m=+0.323960038,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-141.ec2.internal,}"
Jul 18 01:50:31.045352 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045329    2560 manager.go:337] "Starting Device Plugin manager"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.045425    2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045440    2560 server.go:87] "Starting device plugin registration server"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045738    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045748    2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045844    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045940    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.045947    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.046495    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 18 01:50:31.062390 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.046569    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.108168 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.108130    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 18 01:50:31.109628 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.109611    2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 18 01:50:31.109701 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.109651    2560 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 18 01:50:31.109701 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.109680    2560 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 18 01:50:31.109775 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.109714    2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 18 01:50:31.111857 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.111835    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:31.146091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.146063    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.147129 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.147107    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.147234 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.147141    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.147234 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.147151    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.147234 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.147175    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.153548 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.153464    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.153548 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.153495    2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-141.ec2.internal\": node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.181756 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.181725    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.210689 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.210652    2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"]
Jul 18 01:50:31.210848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.210734    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.211742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.211722    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.211814 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.211762    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.211814 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.211776    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.214083 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.214067    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.214202 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.214186    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.214263 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.214222    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.215429 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215409    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.215559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215444    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.215559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215459    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.216014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215757    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.216014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215791    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.216014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.215805    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.219370 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.219352    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.219453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.219381    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:31.220233 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.220218    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:31.220301 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.220246    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:31.220301 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.220256    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:31.249431 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.249401    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-141.ec2.internal\" not found" node="ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.253870 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.253851    2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-141.ec2.internal\" not found" node="ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.282355 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.282326    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.283505 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.283489    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.283579 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.283526    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.283579 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.283546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b26391ef7c32d7e62df830c550d1274c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-141.ec2.internal\" (UID: \"b26391ef7c32d7e62df830c550d1274c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.383163 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.383129    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.384277 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384255    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.384333 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.384333 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384301    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.384333 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384320    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b26391ef7c32d7e62df830c550d1274c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-141.ec2.internal\" (UID: \"b26391ef7c32d7e62df830c550d1274c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.384437 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384355    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b26391ef7c32d7e62df830c550d1274c-config\") pod \"kube-apiserver-proxy-ip-10-0-129-141.ec2.internal\" (UID: \"b26391ef7c32d7e62df830c550d1274c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.384437 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.384385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b6969ccf583981475819b84e43e26d0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal\" (UID: \"8b6969ccf583981475819b84e43e26d0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.484037 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.483966    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.551254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.551220    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.558090 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.558072    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:31.584485 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.584455    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.685059 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.685019    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.785649 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.785562    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.806326 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.806299    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:31.885689 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.885654    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:31.926932 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.926898    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 18 01:50:31.927347 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.927051    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:50:31.927347 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:31.927083    2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 18 01:50:31.986062 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:31.986026    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:32.017034 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.017002    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-17 01:45:31 +0000 UTC" deadline="2028-01-23 14:34:41.711242054 +0000 UTC"
Jul 18 01:50:32.017034 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.017034    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13308h44m9.694211748s"
Jul 18 01:50:32.071379 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.071287    2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 18 01:50:32.072657 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.072625    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientMemory"
Jul 18 01:50:32.072842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.072665    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasNoDiskPressure"
Jul 18 01:50:32.072842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.072679    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeHasSufficientPID"
Jul 18 01:50:32.086627 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.086596    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:32.187278 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.187250    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:32.192628 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:32.192600    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6969ccf583981475819b84e43e26d0.slice/crio-7bbdc9f69b50f0490705dd3f03bc9df177f58f636b813df4d5c66ed59ed23b3b WatchSource:0}: Error finding container 7bbdc9f69b50f0490705dd3f03bc9df177f58f636b813df4d5c66ed59ed23b3b: Status 404 returned error can't find the container with id 7bbdc9f69b50f0490705dd3f03bc9df177f58f636b813df4d5c66ed59ed23b3b
Jul 18 01:50:32.200576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.200559    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:50:32.203869 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:32.203846    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb26391ef7c32d7e62df830c550d1274c.slice/crio-88aa6141cbfb095934386645498d94faf2e5d017c77138fe56b38f8d31d2e8f3 WatchSource:0}: Error finding container 88aa6141cbfb095934386645498d94faf2e5d017c77138fe56b38f8d31d2e8f3: Status 404 returned error can't find the container with id 88aa6141cbfb095934386645498d94faf2e5d017c77138fe56b38f8d31d2e8f3
Jul 18 01:50:32.287672 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.287631    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:32.388172 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.388142    2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-141.ec2.internal\" not found"
Jul 18 01:50:32.476210 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.476188    2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:32.481364 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.481351    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:32.490840 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.490814    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:50:32.491727 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.491713    2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"
Jul 18 01:50:32.505275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.505256    2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:32.505742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.505727    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 18 01:50:32.959713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.959616    2560 apiserver.go:52] "Watching apiserver"
Jul 18 01:50:32.967617 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.966572    2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 18 01:50:32.967617 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.967190    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-gmx26","kube-system/global-pull-secret-syncer-jcgp4","kube-system/konnectivity-agent-zmbg6","kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal","openshift-multus/multus-l7sd8","openshift-multus/network-metrics-daemon-6r66r","openshift-network-diagnostics/network-check-target-fflbk","openshift-network-operator/iptables-alerter-znzw9","openshift-ovn-kubernetes/ovnkube-node-jb5hg","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4","openshift-cluster-node-tuning-operator/tuned-hqkd6","openshift-dns/node-resolver-qcblm","openshift-image-registry/node-ca-q7wjb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal"]
Jul 18 01:50:32.970182 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.970156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.972998 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.972972    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 18 01:50:32.973110 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.972977    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 18 01:50:32.974404 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.973728    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.974855    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.974856    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mnlgm\""
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.975952    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.976093    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.976250    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zxfjb\""
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.976458    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 18 01:50:32.977294 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.976655    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 18 01:50:32.979337 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.978639    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:32.979814 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.979726    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 18 01:50:32.981996 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.980113    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 18 01:50:32.981996 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.980544    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g6b92\""
Jul 18 01:50:32.981996 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.981210    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 18 01:50:32.981996 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.981744    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 18 01:50:32.982252 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.982046    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5z5hx\""
Jul 18 01:50:32.982339 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.982322    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:50:32.984628 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.982665    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.984628 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.983458    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 18 01:50:32.984628 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.984391    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:32.984628 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.984461    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:32.985311 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.985278    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 18 01:50:32.985478 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.985404    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 18 01:50:32.985478 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.985450    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 18 01:50:32.985478 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.985464    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-4k2cq\""
Jul 18 01:50:32.985883 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.985866    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 18 01:50:32.987328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.986747    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:32.988900 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.988839    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 18 01:50:32.989815 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.989794    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 18 01:50:32.990046 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.990025    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 18 01:50:32.990263 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.990244    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4747d\""
Jul 18 01:50:32.990467 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.990445    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:32.990586 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.990545    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:32.990685 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.990664    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 18 01:50:32.991328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991305    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 18 01:50:32.991447 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991364    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 18 01:50:32.991782 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991759    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:32.991877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991795    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-device-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.991877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991820    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-etc-kubernetes\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.991877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991847    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5gh5\" (UniqueName: \"kubernetes.io/projected/a1601f09-60c2-482f-bd62-846866d5bbdf-kube-api-access-f5gh5\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991875    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-registration-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991905    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh9x4\" (UniqueName: \"kubernetes.io/projected/bfa98480-0d80-472c-897d-a52639837b5e-kube-api-access-hh9x4\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991932    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-socket-dir-parent\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991955    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-netns\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.991985    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-conf-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992043 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992027    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-etc-selinux\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992053    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992078    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-socket-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf8x7\" (UniqueName: \"kubernetes.io/projected/d1937935-415d-4a50-b576-c9e5c6561e33-kube-api-access-xf8x7\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992133    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-cnibin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-cni-binary-copy\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992203    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7e01843-2427-4a79-946a-d0ab277a748b-agent-certs\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7e01843-2427-4a79-946a-d0ab277a748b-konnectivity-ca\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa98480-0d80-472c-897d-a52639837b5e-iptables-alerter-script\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992363    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-os-release\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992385    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-hostroot\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.992949 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992420    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-sys-fs\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:32.992949 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-host\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:32.992949 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa98480-0d80-472c-897d-a52639837b5e-host-slash\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:32.992949 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.992504    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-system-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994359    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-multus\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7b4r\" (UniqueName: \"kubernetes.io/projected/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-kube-api-access-w7b4r\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-k8s-cni-cncf-io\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-bin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-kubelet\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994538    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-daemon-config\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.994753    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:32.995847 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:32.994846    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:32.997540 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.996371    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.000460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.999585    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-multus-certs\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.000460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.999658    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.000460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.999695    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-serviceca\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.000460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:32.999728    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txngz\" (UniqueName: \"kubernetes.io/projected/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-kube-api-access-txngz\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.006274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.003735    2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 18 01:50:33.006274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.003941    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 18 01:50:33.006274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.004018    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 18 01:50:33.006274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.004239    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lggf4\""
Jul 18 01:50:33.006274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.004669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.007651 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.007199    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 18 01:50:33.007651 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.007452    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5lgtk\""
Jul 18 01:50:33.007651 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.007492    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:50:33.007950 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.007930    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.010894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.010218    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 18 01:50:33.010894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.010412    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k6ttz\""
Jul 18 01:50:33.010894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.010801    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 18 01:50:33.025726 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.023993    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs29t"
Jul 18 01:50:33.032309 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.032264    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bs29t"
Jul 18 01:50:33.082906 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.082875    2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.099968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100017    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-socket-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-kubelet-config\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100080    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-slash\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100100    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-log-socket\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100125    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-bin\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100152    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-script-lib\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-systemd\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100212    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-cni-binary-copy\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-dbus\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100260    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100293    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-modprobe-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100318    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trgxb\" (UniqueName: \"kubernetes.io/projected/c2372ca7-a57c-491e-be89-b3f4c5277758-kube-api-access-trgxb\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa98480-0d80-472c-897d-a52639837b5e-iptables-alerter-script\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.102576 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100408    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-os-release\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100442    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-sys-fs\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100462    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-host\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100484    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa98480-0d80-472c-897d-a52639837b5e-host-slash\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100504    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-system-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100527    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-multus\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-var-lib-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100591    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-bin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-kubelet\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100675    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-multus-certs\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100705    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100766    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-tmp\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-kubelet\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-host\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100872    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93de963b-7c22-4e91-a404-82baa95cafc7-hosts-file\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.103531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-netns\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100959    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.100990    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-etc-kubernetes\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101016    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5gh5\" (UniqueName: \"kubernetes.io/projected/a1601f09-60c2-482f-bd62-846866d5bbdf-kube-api-access-f5gh5\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-registration-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101070    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-socket-dir-parent\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101110    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-etc-selinux\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101142    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-run\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101165    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-sys\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-node-log\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101219    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101245    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-os-release\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-systemd-units\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101300    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf8x7\" (UniqueName: \"kubernetes.io/projected/d1937935-415d-4a50-b576-c9e5c6561e33-kube-api-access-xf8x7\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101327    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-system-cni-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101353    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-var-lib-kubelet\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-tuned\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.104396 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101404    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-ovn\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101439    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-cnibin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101466    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7e01843-2427-4a79-946a-d0ab277a748b-agent-certs\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101494    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7e01843-2427-4a79-946a-d0ab277a748b-konnectivity-ca\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101546    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93de963b-7c22-4e91-a404-82baa95cafc7-tmp-dir\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101619    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysconfig\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101645    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-kubernetes\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101672    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-hostroot\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101699    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-cnibin\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101725    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-config\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-env-overrides\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101766    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7b4r\" (UniqueName: \"kubernetes.io/projected/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-kube-api-access-w7b4r\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-lib-modules\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p462\" (UniqueName: \"kubernetes.io/projected/93de963b-7c22-4e91-a404-82baa95cafc7-kube-api-access-8p462\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101843    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-etc-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.105259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101861    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-netd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101907    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-k8s-cni-cncf-io\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101924    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-daemon-config\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101947    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-serviceca\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101974    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txngz\" (UniqueName: \"kubernetes.io/projected/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-kube-api-access-txngz\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.101997    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-conf\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2372ca7-a57c-491e-be89-b3f4c5277758-ovn-node-metrics-cert\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102037    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-device-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-systemd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh9x4\" (UniqueName: \"kubernetes.io/projected/bfa98480-0d80-472c-897d-a52639837b5e-kube-api-access-hh9x4\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102132    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-netns\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102155    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-conf-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102186    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmxqb\" (UniqueName: \"kubernetes.io/projected/6facbd29-cddd-4b48-80bc-76682898ab95-kube-api-access-bmxqb\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102205    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdn4j\" (UniqueName: \"kubernetes.io/projected/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-kube-api-access-wdn4j\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.106176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102522    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-socket-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102658    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-k8s-cni-cncf-io\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.102712    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-cnibin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.103077    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.103262    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-daemon-config\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.103743    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-serviceca\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.104127    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.104195    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:50:33.604172683 +0000 UTC m=+2.926552851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.104476    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-device-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.104736    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-netns\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.104786    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-conf-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.104912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105525    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bfa98480-0d80-472c-897d-a52639837b5e-iptables-alerter-script\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105629    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-os-release\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105691    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-host\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105708    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-registration-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bfa98480-0d80-472c-897d-a52639837b5e-host-slash\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105767    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-sys-fs\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.106981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105820    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-system-cni-dir\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105860    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-etc-kubernetes\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105872    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-run-multus-certs\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-multus\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105948    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-multus-socket-dir-parent\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.105971    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-cni-bin\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.106034    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-etc-selinux\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.106085    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-host-var-lib-kubelet\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.106336    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d1937935-415d-4a50-b576-c9e5c6561e33-kubelet-dir\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.106382    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a1601f09-60c2-482f-bd62-846866d5bbdf-hostroot\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.106552    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d7e01843-2427-4a79-946a-d0ab277a748b-konnectivity-ca\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.107058    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a1601f09-60c2-482f-bd62-846866d5bbdf-cni-binary-copy\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.107803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.107331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d7e01843-2427-4a79-946a-d0ab277a748b-agent-certs\") pod \"konnectivity-agent-zmbg6\" (UID: \"d7e01843-2427-4a79-946a-d0ab277a748b\") " pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:33.114893 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.114870    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txngz\" (UniqueName: \"kubernetes.io/projected/ac9788af-d9b4-4af3-8220-d2b97fdc56d0-kube-api-access-txngz\") pod \"node-ca-q7wjb\" (UID: \"ac9788af-d9b4-4af3-8220-d2b97fdc56d0\") " pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.116177 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.116116    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh9x4\" (UniqueName: \"kubernetes.io/projected/bfa98480-0d80-472c-897d-a52639837b5e-kube-api-access-hh9x4\") pod \"iptables-alerter-znzw9\" (UID: \"bfa98480-0d80-472c-897d-a52639837b5e\") " pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.116305 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.116234    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal" event={"ID":"b26391ef7c32d7e62df830c550d1274c","Type":"ContainerStarted","Data":"88aa6141cbfb095934386645498d94faf2e5d017c77138fe56b38f8d31d2e8f3"}
Jul 18 01:50:33.118395 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.118371    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5gh5\" (UniqueName: \"kubernetes.io/projected/a1601f09-60c2-482f-bd62-846866d5bbdf-kube-api-access-f5gh5\") pod \"multus-l7sd8\" (UID: \"a1601f09-60c2-482f-bd62-846866d5bbdf\") " pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.119484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.119413    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal" event={"ID":"8b6969ccf583981475819b84e43e26d0","Type":"ContainerStarted","Data":"7bbdc9f69b50f0490705dd3f03bc9df177f58f636b813df4d5c66ed59ed23b3b"}
Jul 18 01:50:33.121658 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.121636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf8x7\" (UniqueName: \"kubernetes.io/projected/d1937935-415d-4a50-b576-c9e5c6561e33-kube-api-access-xf8x7\") pod \"aws-ebs-csi-driver-node-js2r4\" (UID: \"d1937935-415d-4a50-b576-c9e5c6561e33\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.122127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.122102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7b4r\" (UniqueName: \"kubernetes.io/projected/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-kube-api-access-w7b4r\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:33.202533 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-var-lib-kubelet\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202533 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202520    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-tuned\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-ovn\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202597    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.202775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202622    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-var-lib-kubelet\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202626    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93de963b-7c22-4e91-a404-82baa95cafc7-tmp-dir\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202803    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-ovn\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysconfig\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202859    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-kubernetes\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202861    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202888    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-cnibin\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202912    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysconfig\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202914    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-config\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93de963b-7c22-4e91-a404-82baa95cafc7-tmp-dir\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-env-overrides\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.202993 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202995    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-lib-modules\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203022    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p462\" (UniqueName: \"kubernetes.io/projected/93de963b-7c22-4e91-a404-82baa95cafc7-kube-api-access-8p462\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203029    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-kubernetes\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203049    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-etc-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.202995    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-cnibin\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203074    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203098    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-netd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-conf\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203150    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2372ca7-a57c-491e-be89-b3f4c5277758-ovn-node-metrics-cert\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203171    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-lib-modules\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203189    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-systemd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-ovn-kubernetes\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203219    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmxqb\" (UniqueName: \"kubernetes.io/projected/6facbd29-cddd-4b48-80bc-76682898ab95-kube-api-access-bmxqb\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203262    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdn4j\" (UniqueName: \"kubernetes.io/projected/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-kube-api-access-wdn4j\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203289    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-kubelet-config\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203348    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.203559 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203378    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-slash\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203399    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-log-socket\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203422    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-bin\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203429    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-env-overrides\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203447    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-script-lib\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203472    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-systemd\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-netd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203499    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-dbus\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203525    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203551    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-cni-bin\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203552    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-modprobe-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203586    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-conf\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203500    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-config\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-systemd\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-log-socket\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203666    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trgxb\" (UniqueName: \"kubernetes.io/projected/c2372ca7-a57c-491e-be89-b3f4c5277758-kube-api-access-trgxb\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203703    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-var-lib-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.204484 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203736    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203787    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-tmp\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203812    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-kubelet\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203839    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-host\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203865    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93de963b-7c22-4e91-a404-82baa95cafc7-hosts-file\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203890    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-netns\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203951    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-run\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203977    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-sys\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.203983    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-etc-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204002    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-node-log\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204056    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-os-release\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204055    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204072    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204094    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-systemd-units\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.205516 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-system-cni-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204170    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-modprobe-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-host\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204192    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-system-cni-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-slash\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204299    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-os-release\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c2372ca7-a57c-491e-be89-b3f4c5277758-ovnkube-script-lib\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204344    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-kubelet-config\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204387    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-systemd\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204393    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-systemd-units\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204435    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-run-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204442    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-run\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204455    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/93de963b-7c22-4e91-a404-82baa95cafc7-hosts-file\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204484    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-run-netns\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204615    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-sysctl-d\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204647    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6facbd29-cddd-4b48-80bc-76682898ab95-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.204707    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204702    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-host-kubelet\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.206189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204745    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-var-lib-openvswitch\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.207139 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c2372ca7-a57c-491e-be89-b3f4c5277758-node-log\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.207139 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.204767    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:50:33.704747818 +0000 UTC m=+3.027127969 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:33.207139 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204702    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-sys\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.207139 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-dbus\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.207139 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.204967    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6facbd29-cddd-4b48-80bc-76682898ab95-cni-binary-copy\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.208051 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.208027    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c2372ca7-a57c-491e-be89-b3f4c5277758-ovn-node-metrics-cert\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.208128 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.208028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-tmp\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.209922 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.209852    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:33.209922 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.209880    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:33.209922 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.209893    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:33.210287 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.209958    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:50:33.709942242 +0000 UTC m=+3.032322412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:33.213469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.212718    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p462\" (UniqueName: \"kubernetes.io/projected/93de963b-7c22-4e91-a404-82baa95cafc7-kube-api-access-8p462\") pod \"node-resolver-qcblm\" (UID: \"93de963b-7c22-4e91-a404-82baa95cafc7\") " pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.213469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.213082    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdn4j\" (UniqueName: \"kubernetes.io/projected/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-kube-api-access-wdn4j\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.213469 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.213435    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmxqb\" (UniqueName: \"kubernetes.io/projected/6facbd29-cddd-4b48-80bc-76682898ab95-kube-api-access-bmxqb\") pod \"multus-additional-cni-plugins-gmx26\" (UID: \"6facbd29-cddd-4b48-80bc-76682898ab95\") " pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.215950 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.215923    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trgxb\" (UniqueName: \"kubernetes.io/projected/c2372ca7-a57c-491e-be89-b3f4c5277758-kube-api-access-trgxb\") pod \"ovnkube-node-jb5hg\" (UID: \"c2372ca7-a57c-491e-be89-b3f4c5277758\") " pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.216092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.216064    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/13a00b39-43b8-45f9-8e46-ff9b9b27a6be-etc-tuned\") pod \"tuned-hqkd6\" (UID: \"13a00b39-43b8-45f9-8e46-ff9b9b27a6be\") " pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.226192 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.226174    2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:33.290976 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.290928    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4"
Jul 18 01:50:33.306452 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.306424    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q7wjb"
Jul 18 01:50:33.316438 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.316404    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:33.325536 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.325486    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-znzw9"
Jul 18 01:50:33.333954 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.333932    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-l7sd8"
Jul 18 01:50:33.343342 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.343317    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:50:33.352675 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.352648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gmx26"
Jul 18 01:50:33.362917 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.362894    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hqkd6"
Jul 18 01:50:33.368970 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.368943    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qcblm"
Jul 18 01:50:33.607514 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.607407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:33.607666 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.607608    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:33.607716 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.607673    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:50:34.607657245 +0000 UTC m=+3.930037392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:33.708472 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.708430    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:33.708689 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.708630    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:33.708758 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.708711    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:50:34.708695314 +0000 UTC m=+4.031075460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:33.809156 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:33.809113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:33.809358 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.809265    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:33.809358 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.809288    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:33.809358 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.809303    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:33.809490 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:33.809374    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:50:34.809354014 +0000 UTC m=+4.131734167 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:34.008362 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.008324    2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 18 01:50:34.016998 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.016966    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1937935_415d_4a50_b576_c9e5c6561e33.slice/crio-a268028ea43cc09df03e666ee2ef8532a5818ac74d7d106830521881c5b557da WatchSource:0}: Error finding container a268028ea43cc09df03e666ee2ef8532a5818ac74d7d106830521881c5b557da: Status 404 returned error can't find the container with id a268028ea43cc09df03e666ee2ef8532a5818ac74d7d106830521881c5b557da
Jul 18 01:50:34.038122 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.038085    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:45:33 +0000 UTC" deadline="2028-03-11 17:53:27.738569316 +0000 UTC"
Jul 18 01:50:34.038122 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.038118    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14464h2m53.700455418s"
Jul 18 01:50:34.110957 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.110923    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:34.111153 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.111063    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:34.122404 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.122347    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4" event={"ID":"d1937935-415d-4a50-b576-c9e5c6561e33","Type":"ContainerStarted","Data":"a268028ea43cc09df03e666ee2ef8532a5818ac74d7d106830521881c5b557da"}
Jul 18 01:50:34.377569 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.377518    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6facbd29_cddd_4b48_80bc_76682898ab95.slice/crio-91382b19d91b24d18304deb5bdfbee118950be7bdb3a04c7a51b2ad2aeff146f WatchSource:0}: Error finding container 91382b19d91b24d18304deb5bdfbee118950be7bdb3a04c7a51b2ad2aeff146f: Status 404 returned error can't find the container with id 91382b19d91b24d18304deb5bdfbee118950be7bdb3a04c7a51b2ad2aeff146f
Jul 18 01:50:34.380579 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.380554    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa98480_0d80_472c_897d_a52639837b5e.slice/crio-e92d84952631b91a6eb77fb1a378cb632048c2cbea789effa13d1f287b42f60e WatchSource:0}: Error finding container e92d84952631b91a6eb77fb1a378cb632048c2cbea789effa13d1f287b42f60e: Status 404 returned error can't find the container with id e92d84952631b91a6eb77fb1a378cb632048c2cbea789effa13d1f287b42f60e
Jul 18 01:50:34.385171 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.385132    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac9788af_d9b4_4af3_8220_d2b97fdc56d0.slice/crio-4ea029d2e86614c3f6d1416ed0ec5c01f32ccdbab821da5d5e2f5def9b915711 WatchSource:0}: Error finding container 4ea029d2e86614c3f6d1416ed0ec5c01f32ccdbab821da5d5e2f5def9b915711: Status 404 returned error can't find the container with id 4ea029d2e86614c3f6d1416ed0ec5c01f32ccdbab821da5d5e2f5def9b915711
Jul 18 01:50:34.386301 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.386274    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93de963b_7c22_4e91_a404_82baa95cafc7.slice/crio-a2ae787bb49158ea47f9c4735bccee7bce5a782bcb33cca9edcf5f50989e2e71 WatchSource:0}: Error finding container a2ae787bb49158ea47f9c4735bccee7bce5a782bcb33cca9edcf5f50989e2e71: Status 404 returned error can't find the container with id a2ae787bb49158ea47f9c4735bccee7bce5a782bcb33cca9edcf5f50989e2e71
Jul 18 01:50:34.387468 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.387273    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2372ca7_a57c_491e_be89_b3f4c5277758.slice/crio-e3f24246637befb36bdba1f3b228ab0aef6ca953aca918c8e78f879d4f84731f WatchSource:0}: Error finding container e3f24246637befb36bdba1f3b228ab0aef6ca953aca918c8e78f879d4f84731f: Status 404 returned error can't find the container with id e3f24246637befb36bdba1f3b228ab0aef6ca953aca918c8e78f879d4f84731f
Jul 18 01:50:34.388405 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.388369    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1601f09_60c2_482f_bd62_846866d5bbdf.slice/crio-03344a7953b09ec4c111d561d50fad9ecfe7346ba6668a2b7ee3c81aae9f047a WatchSource:0}: Error finding container 03344a7953b09ec4c111d561d50fad9ecfe7346ba6668a2b7ee3c81aae9f047a: Status 404 returned error can't find the container with id 03344a7953b09ec4c111d561d50fad9ecfe7346ba6668a2b7ee3c81aae9f047a
Jul 18 01:50:34.394679 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.390480    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e01843_2427_4a79_946a_d0ab277a748b.slice/crio-d5bcf07a75311a3c84387eed58f91c95b2967294a86dfc431579e2c7ccc9f5c5 WatchSource:0}: Error finding container d5bcf07a75311a3c84387eed58f91c95b2967294a86dfc431579e2c7ccc9f5c5: Status 404 returned error can't find the container with id d5bcf07a75311a3c84387eed58f91c95b2967294a86dfc431579e2c7ccc9f5c5
Jul 18 01:50:34.394679 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:50:34.392008    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a00b39_43b8_45f9_8e46_ff9b9b27a6be.slice/crio-7103f7ab8797f34156fc9cfe19076c351142380cfa5f22a37c1b1e9e8a0bf5f3 WatchSource:0}: Error finding container 7103f7ab8797f34156fc9cfe19076c351142380cfa5f22a37c1b1e9e8a0bf5f3: Status 404 returned error can't find the container with id 7103f7ab8797f34156fc9cfe19076c351142380cfa5f22a37c1b1e9e8a0bf5f3
Jul 18 01:50:34.614738 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.614711    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:34.615044 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.614836    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:34.615044 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.614888    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:50:36.614873139 +0000 UTC m=+5.937253288 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:34.715739 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.715696    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:34.715931 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.715889    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:34.715991 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.715978    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:50:36.715955608 +0000 UTC m=+6.038335769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:34.816948 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:34.816904    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:34.817134 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.817104    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:34.817134 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.817130    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:34.817212 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.817142    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:34.817212 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:34.817194    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:50:36.81717731 +0000 UTC m=+6.139557459 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:35.038834 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.038771    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-17 01:45:33 +0000 UTC" deadline="2028-04-13 18:27:45.010724948 +0000 UTC"
Jul 18 01:50:35.038834 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.038832    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15256h37m9.971897853s"
Jul 18 01:50:35.118390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.117678    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:35.118390 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:35.117812    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:35.118390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.118228    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:35.118390 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:35.118336    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:35.149148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.149105    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal" event={"ID":"b26391ef7c32d7e62df830c550d1274c","Type":"ContainerStarted","Data":"65b6ec45315bb416198e824c2fc9190f5c2c75d6bdfa4301cce2896cebb6de93"}
Jul 18 01:50:35.159323 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.159248    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qcblm" event={"ID":"93de963b-7c22-4e91-a404-82baa95cafc7","Type":"ContainerStarted","Data":"a2ae787bb49158ea47f9c4735bccee7bce5a782bcb33cca9edcf5f50989e2e71"}
Jul 18 01:50:35.165087 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.165021    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-141.ec2.internal" podStartSLOduration=2.164998743 podStartE2EDuration="2.164998743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:35.16453655 +0000 UTC m=+4.486916712" watchObservedRunningTime="2026-07-18 01:50:35.164998743 +0000 UTC m=+4.487378911"
Jul 18 01:50:35.173054 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.173010    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q7wjb" event={"ID":"ac9788af-d9b4-4af3-8220-d2b97fdc56d0","Type":"ContainerStarted","Data":"4ea029d2e86614c3f6d1416ed0ec5c01f32ccdbab821da5d5e2f5def9b915711"}
Jul 18 01:50:35.178110 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.178075    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerStarted","Data":"91382b19d91b24d18304deb5bdfbee118950be7bdb3a04c7a51b2ad2aeff146f"}
Jul 18 01:50:35.181711 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.181676    2560 generic.go:356] "Generic (PLEG): container finished" podID="8b6969ccf583981475819b84e43e26d0" containerID="8ebed3565949de461b89ef6fc416dd704d542908b2a61548600cedd161011549" exitCode=0
Jul 18 01:50:35.181866 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.181770    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal" event={"ID":"8b6969ccf583981475819b84e43e26d0","Type":"ContainerDied","Data":"8ebed3565949de461b89ef6fc416dd704d542908b2a61548600cedd161011549"}
Jul 18 01:50:35.187979 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.187939    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zmbg6" event={"ID":"d7e01843-2427-4a79-946a-d0ab277a748b","Type":"ContainerStarted","Data":"d5bcf07a75311a3c84387eed58f91c95b2967294a86dfc431579e2c7ccc9f5c5"}
Jul 18 01:50:35.195289 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.195253    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hqkd6" event={"ID":"13a00b39-43b8-45f9-8e46-ff9b9b27a6be","Type":"ContainerStarted","Data":"7103f7ab8797f34156fc9cfe19076c351142380cfa5f22a37c1b1e9e8a0bf5f3"}
Jul 18 01:50:35.201246 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.201215    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7sd8" event={"ID":"a1601f09-60c2-482f-bd62-846866d5bbdf","Type":"ContainerStarted","Data":"03344a7953b09ec4c111d561d50fad9ecfe7346ba6668a2b7ee3c81aae9f047a"}
Jul 18 01:50:35.214012 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.213970    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"e3f24246637befb36bdba1f3b228ab0aef6ca953aca918c8e78f879d4f84731f"}
Jul 18 01:50:35.222286 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:35.222243    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-znzw9" event={"ID":"bfa98480-0d80-472c-897d-a52639837b5e","Type":"ContainerStarted","Data":"e92d84952631b91a6eb77fb1a378cb632048c2cbea789effa13d1f287b42f60e"}
Jul 18 01:50:36.110155 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:36.110115    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:36.110690 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.110260    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:36.232517 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:36.232480    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal" event={"ID":"8b6969ccf583981475819b84e43e26d0","Type":"ContainerStarted","Data":"220f4d04ffe48b4bc480f73ef1dd545eccbf456a2adc40d268d7a97dd34e7e92"}
Jul 18 01:50:36.638015 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:36.637795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:36.638015 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.637977    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:36.638256 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.638040    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:50:40.638019652 +0000 UTC m=+9.960399813 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:36.738378 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:36.738333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:36.738801 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.738750    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:36.738923 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.738868    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:50:40.738843957 +0000 UTC m=+10.061224105 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:36.840035 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:36.839992    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:36.840237 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.840182    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:36.840237 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.840210    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:36.840237 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.840226    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:36.840420 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:36.840293    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:50:40.840270998 +0000 UTC m=+10.162651145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:37.111615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:37.110042    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:37.111615 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:37.110217    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:37.111615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:37.110681    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:37.111615 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:37.110797    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:38.110817 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:38.110774    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:38.111021 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:38.110926    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:39.110854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:39.110809    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:39.111339 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:39.110937    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:39.111339 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:39.111011    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:39.111339 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:39.111098    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:40.109981 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:40.109936    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:40.110200 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.110085    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:40.670481 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:40.670434    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:40.671030 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.670627    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:40.671030 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.670713    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:50:48.670692992 +0000 UTC m=+17.993073170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:40.771522 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:40.770864    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:40.771522 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.771063    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:40.771522 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.771131    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:50:48.771110165 +0000 UTC m=+18.093490317 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:40.871582 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:40.871529    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:40.871768 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.871726    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:40.871768 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.871743    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:40.871768 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.871753    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:40.871926 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:40.871800    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:50:48.871786338 +0000 UTC m=+18.194166484 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:41.110006 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:41.109905    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:41.110185 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:41.110061    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:41.110185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:41.110126    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:41.110321 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:41.110196    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:42.110852 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:42.110811    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:42.111366 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:42.110958    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:43.110174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:43.110132    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:43.110328 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:43.110265    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:43.110739 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:43.110717    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:43.110869 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:43.110848    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:44.110131 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:44.110095    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:44.110742 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:44.110238    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:45.114345 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:45.113915    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:45.114345 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:45.114035    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:45.114345 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:45.114098    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:45.114345 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:45.114208    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:46.110308 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:46.110270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:46.110396 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:46.110377    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:47.113719 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:47.113689    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:47.114200 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:47.113818    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:47.114200 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:47.113865    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:47.114200 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:47.113971    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:48.110308 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:48.110265    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:48.110571 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.110412    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:48.743377 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:48.743330    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:48.743958 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.743533    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:48.743958 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.743622    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:51:04.743598844 +0000 UTC m=+34.065978988 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:50:48.844715 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:48.844677    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:48.844890 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.844866    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:48.844979 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.844934    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:04.844918196 +0000 UTC m=+34.167298342 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:50:48.945779 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:48.945742    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:48.945980 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.945896    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:50:48.945980 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.945919    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:50:48.945980 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.945931    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:48.946118 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:48.945994    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:51:04.945974746 +0000 UTC m=+34.268354914 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:50:49.115592 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:49.115501    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:49.115762 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:49.115641    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:49.115987 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:49.115969    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:49.116096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:49.116077    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:50.110386 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:50.110345    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:50.110866 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:50.110473    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:51.113744 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:51.113713    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:51.114191 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:51.113818    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:51.114191 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:51.113889    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:51.114191 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:51.114017    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:52.110853 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:52.110814    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:52.111020 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:52.110953    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:53.110124 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:53.110084    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:53.110556 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:53.110124    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:53.110556 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:53.110220    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:53.110556 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:53.110333    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:54.110039 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:54.109993    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:54.110329 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:54.110116    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:55.111089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.110776    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:55.111955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.110794    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:55.111955 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:55.111178    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:55.111955 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:55.111245    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:55.287316 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.287274    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="9139d2d50680ba554cfc8b4a6f12d78f000a01066301d64551c9606b2123cf5e" exitCode=0
Jul 18 01:50:55.287504 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.287382    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"9139d2d50680ba554cfc8b4a6f12d78f000a01066301d64551c9606b2123cf5e"}
Jul 18 01:50:55.290459 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.290432    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zmbg6" event={"ID":"d7e01843-2427-4a79-946a-d0ab277a748b","Type":"ContainerStarted","Data":"baa902561bc828583bbe47eea3bbd26d48ebc0d0e835f5b2696020c0c5f431ee"}
Jul 18 01:50:55.292260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.292188    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hqkd6" event={"ID":"13a00b39-43b8-45f9-8e46-ff9b9b27a6be","Type":"ContainerStarted","Data":"cc57a134f621f2821e76596655d9e8047d10e103518119ac8349bac9b624c9a8"}
Jul 18 01:50:55.293875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.293851    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-l7sd8" event={"ID":"a1601f09-60c2-482f-bd62-846866d5bbdf","Type":"ContainerStarted","Data":"cdf6c4a3f268f61176edb1e704d06f576c3ba3c707cad44432a56e2d60ea2d24"}
Jul 18 01:50:55.297582 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297553    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"1a2f96a9a4d7d036066754a485196ab6cf130138c8be423e43abf354b0807643"}
Jul 18 01:50:55.297686 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297624    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"74fc1f8c158c77516634a5ae1000f000c09a55b37efd413ec4508cae32ff84db"}
Jul 18 01:50:55.297686 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"c9292254b3a798543e570b07e7b5a8aa278dadff0b2b1b51561e9e839a03e852"}
Jul 18 01:50:55.297777 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297698    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"f667e8fcd7c1d7ed505ecc01aaea062479c046be803549ad373e2e71f8afdb6b"}
Jul 18 01:50:55.297777 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297729    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"86182e355fd924bbeccb2ae8ec66624e7ee7de201ab88330d16eade2042e4f5c"}
Jul 18 01:50:55.297777 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.297763    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"44846e5f4c8916c66ac3313f3cd6b9d93c5e3bddbbca7a02af66d6dd3512ee0e"}
Jul 18 01:50:55.299080 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.299058    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4" event={"ID":"d1937935-415d-4a50-b576-c9e5c6561e33","Type":"ContainerStarted","Data":"fa3cf3903fd6d0d565b8b3e7c5b012a16515b2d4bb3d9cf05a8cde9a1c06c496"}
Jul 18 01:50:55.300570 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.300544    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qcblm" event={"ID":"93de963b-7c22-4e91-a404-82baa95cafc7","Type":"ContainerStarted","Data":"af7cd629a6f8df32a76cdea4392a1b483ba15b1d21cba6d53182d1c89e5013fa"}
Jul 18 01:50:55.301872 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.301852    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q7wjb" event={"ID":"ac9788af-d9b4-4af3-8220-d2b97fdc56d0","Type":"ContainerStarted","Data":"da3c2420bb16308ff97fbc501a54c19f6c310c447644d530b8d4b0dcc794e792"}
Jul 18 01:50:55.309382 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.309321    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-141.ec2.internal" podStartSLOduration=22.309304058 podStartE2EDuration="22.309304058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:36.24998529 +0000 UTC m=+5.572365461" watchObservedRunningTime="2026-07-18 01:50:55.309304058 +0000 UTC m=+24.631684224"
Jul 18 01:50:55.346695 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.346644    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hqkd6" podStartSLOduration=3.5563847920000002 podStartE2EDuration="23.346629433s" totalImagesPullingTime="19.790244641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:55.327452888 +0000 UTC m=+24.649833065" watchObservedRunningTime="2026-07-18 01:50:55.346629433 +0000 UTC m=+24.669009599"
Jul 18 01:50:55.346927 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.346906    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zmbg6" podStartSLOduration=3.578502899 podStartE2EDuration="23.346899759s" totalImagesPullingTime="19.76839686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:55.346562067 +0000 UTC m=+24.668942232" watchObservedRunningTime="2026-07-18 01:50:55.346899759 +0000 UTC m=+24.669279925"
Jul 18 01:50:55.384859 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.384806    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-l7sd8" podStartSLOduration=3.53440112 podStartE2EDuration="23.38478557s" totalImagesPullingTime="19.85038445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:55.368841506 +0000 UTC m=+24.691221729" watchObservedRunningTime="2026-07-18 01:50:55.38478557 +0000 UTC m=+24.707165735"
Jul 18 01:50:55.385284 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.385259    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qcblm" podStartSLOduration=3.610392767 podStartE2EDuration="23.385251607s" totalImagesPullingTime="19.77485884s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:55.384476187 +0000 UTC m=+24.706856372" watchObservedRunningTime="2026-07-18 01:50:55.385251607 +0000 UTC m=+24.707631773"
Jul 18 01:50:55.398861 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:55.398811    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q7wjb" podStartSLOduration=3.599810427 podStartE2EDuration="23.398793717s" totalImagesPullingTime="19.79898329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:55.398702197 +0000 UTC m=+24.721082365" watchObservedRunningTime="2026-07-18 01:50:55.398793717 +0000 UTC m=+24.721173882"
Jul 18 01:50:56.110458 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:56.110419    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:56.110662 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:56.110576    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:56.207253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:56.207219    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 18 01:50:56.306305 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:56.306204    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-znzw9" event={"ID":"bfa98480-0d80-472c-897d-a52639837b5e","Type":"ContainerStarted","Data":"e6250aa8b6a6332fa30623e03a82acbe797aa7b239681719810c72f9ea5ab614"}
Jul 18 01:50:56.308779 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:56.308727    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4" event={"ID":"d1937935-415d-4a50-b576-c9e5c6561e33","Type":"ContainerStarted","Data":"fd49e3955eeeb6118c8a7386e54282e5219c465eaa3593130d0b308d13e6ebba"}
Jul 18 01:50:56.319155 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:56.319106    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-znzw9" podStartSLOduration=4.518164339 podStartE2EDuration="24.319088801s" totalImagesPullingTime="19.800924462s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:56.318919508 +0000 UTC m=+25.641299674" watchObservedRunningTime="2026-07-18 01:50:56.319088801 +0000 UTC m=+25.641469014"
Jul 18 01:50:57.058725 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.058630    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-18T01:50:56.207243233Z","UUID":"e6f829b3-c4cb-41bd-971a-18fcc665e65b","Handler":null,"Name":"","Endpoint":""}
Jul 18 01:50:57.060170 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.060147    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 18 01:50:57.060297 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.060196    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 18 01:50:57.111575 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.111541    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:57.111717 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.111592    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:57.111717 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:57.111685    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:57.111837 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:57.111771    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:50:57.315475 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.315387    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"ab841cfadef8df87a0416efccda7e37db99a83625be79a840f4c82562c3836cd"}
Jul 18 01:50:57.535722 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.535681    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:57.536304 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:57.536281    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:58.110488 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:58.110401    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:50:58.110667 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:58.110535    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:50:58.320537 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:58.320477    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4" event={"ID":"d1937935-415d-4a50-b576-c9e5c6561e33","Type":"ContainerStarted","Data":"fb4e297a6497e6eebb6bd259a4c95f484865809c5d06ea07ceb539a4c8a41a0d"}
Jul 18 01:50:58.321012 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:58.320781    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:58.321333 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:58.321308    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zmbg6"
Jul 18 01:50:58.336983 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:58.336759    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-js2r4" podStartSLOduration=3.054003693 podStartE2EDuration="26.336727325s" totalImagesPullingTime="23.282723632s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:50:58.336699235 +0000 UTC m=+27.659079400" watchObservedRunningTime="2026-07-18 01:50:58.336727325 +0000 UTC m=+27.659107490"
Jul 18 01:50:59.110506 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:59.110474    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:50:59.110506 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:50:59.110489    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:50:59.110763 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:59.110642    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:50:59.110763 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:50:59.110739    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:51:00.110503 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.110472    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:00.111154 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:00.110593    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:51:00.328740 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.328346    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" event={"ID":"c2372ca7-a57c-491e-be89-b3f4c5277758","Type":"ContainerStarted","Data":"7c2a88bec822c3f6295e1d8ccb2a7777a98af037a7a518c8ef087fc7aa354ccf"}
Jul 18 01:51:00.328740 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.328680    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:00.328740 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.328736    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:00.328950 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.328793    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:00.346449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.346420    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:00.346631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.346611    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:00.352982 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:00.352926    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg" podStartSLOduration=8.500229553 podStartE2EDuration="28.352912927s" totalImagesPullingTime="19.852683374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:00.352537687 +0000 UTC m=+29.674917866" watchObservedRunningTime="2026-07-18 01:51:00.352912927 +0000 UTC m=+29.675293093"
Jul 18 01:51:01.111949 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:01.111908    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:01.112460 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:01.112047    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:51:01.112460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:01.112124    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:01.112460 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:01.112305    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:51:01.336818 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:01.336781    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="867c3cc06ef1bb6a0394b0d5077fe9fab24a650f4cfdd4d815b9fc164cf633e5" exitCode=0
Jul 18 01:51:01.337047 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:01.336872    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"867c3cc06ef1bb6a0394b0d5077fe9fab24a650f4cfdd4d815b9fc164cf633e5"}
Jul 18 01:51:02.109901 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.109861    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:02.110105 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:02.109993    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:51:02.299154 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.299119    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fflbk"]
Jul 18 01:51:02.302320 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.302284    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jcgp4"]
Jul 18 01:51:02.302463 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.302405    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:02.302583 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:02.302556    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:51:02.303082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.303050    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6r66r"]
Jul 18 01:51:02.303168 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.303149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:02.303261 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:02.303242    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:51:02.339545 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:02.339498    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:02.339717 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:02.339676    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:51:03.344295 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:03.344259    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="8af36dcab99e91d737964d683abefb460169930b346ef9fc4d1bfe3954a360ce" exitCode=0
Jul 18 01:51:03.344728 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:03.344327    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"8af36dcab99e91d737964d683abefb460169930b346ef9fc4d1bfe3954a360ce"}
Jul 18 01:51:04.110242 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.110010    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:04.110392 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.110013    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:04.110392 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.110266    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:51:04.110392 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.110017    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:04.110392 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.110338    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:51:04.110541 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.110410    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:51:04.758638 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.758540    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:04.759005 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.758695    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:51:04.759005 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.758756    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:51:36.758740814 +0000 UTC m=+66.081120960 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 18 01:51:04.859301 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.859259    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:04.859470 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.859421    2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 18 01:51:04.859506 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.859489    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret podName:8ea73327-3d50-4b1a-b81e-8b49f3c91f73 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:36.859470548 +0000 UTC m=+66.181850709 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret") pod "global-pull-secret-syncer-jcgp4" (UID: "8ea73327-3d50-4b1a-b81e-8b49f3c91f73") : object "kube-system"/"original-pull-secret" not registered
Jul 18 01:51:04.960540 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:04.960485    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:04.960682 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.960641    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 18 01:51:04.960682 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.960665    2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 18 01:51:04.960682 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.960675    2560 projected.go:196] Error preparing data for projected volume kube-api-access-d8xld for pod openshift-network-diagnostics/network-check-target-fflbk: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:51:04.960813 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:04.960727    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld podName:c3f7a9a9-a831-4d43-be7e-cc3a4b84560a nodeName:}" failed. No retries permitted until 2026-07-18 01:51:36.960712236 +0000 UTC m=+66.283092380 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-d8xld" (UniqueName: "kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld") pod "network-check-target-fflbk" (UID: "c3f7a9a9-a831-4d43-be7e-cc3a4b84560a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 18 01:51:05.351679 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:05.351639    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="bd74ba20fd07fd2f525b2905766b327d250281fa89d717bcc7132841d77f0455" exitCode=0
Jul 18 01:51:05.351842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:05.351680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"bd74ba20fd07fd2f525b2905766b327d250281fa89d717bcc7132841d77f0455"}
Jul 18 01:51:06.110492 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:06.110454    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:06.111040 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:06.110585    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:06.111040 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:06.110596    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:06.111040 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:06.110590    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:51:06.111040 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:06.110682    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-jcgp4" podUID="8ea73327-3d50-4b1a-b81e-8b49f3c91f73"
Jul 18 01:51:06.111040 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:06.110738    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fflbk" podUID="c3f7a9a9-a831-4d43-be7e-cc3a4b84560a"
Jul 18 01:51:07.021858 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.021783    2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-141.ec2.internal" event="NodeReady"
Jul 18 01:51:07.022007 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.021929    2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 18 01:51:07.067395 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.066588    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:51:07.070605 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.070568    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"]
Jul 18 01:51:07.070747 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.070707    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.073142 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.073121    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dwp4g\""
Jul 18 01:51:07.073258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.073176    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 18 01:51:07.073449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.073435    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 18 01:51:07.073549 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.073450    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 18 01:51:07.074666 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.074648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.077108 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.077081    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 18 01:51:07.077230 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.077132    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 18 01:51:07.077443 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.077426    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2nvr5\""
Jul 18 01:51:07.079149 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.078932    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 18 01:51:07.080410 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.080388    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tltqd"]
Jul 18 01:51:07.084390 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.084369    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5dlq8"]
Jul 18 01:51:07.084552 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.084505    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.088562 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.088487    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"]
Jul 18 01:51:07.091003 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.090976    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:51:07.091470 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.091451    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 18 01:51:07.091779 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.091763    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qqc6n\""
Jul 18 01:51:07.092023 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.092006    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 18 01:51:07.092395 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.092374    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.095710 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.095674    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bmfns\""
Jul 18 01:51:07.095842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.095765    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 18 01:51:07.095907 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.095679    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 18 01:51:07.096016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.095996    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 18 01:51:07.096350 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.096324    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tltqd"]
Jul 18 01:51:07.100955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.100924    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dlq8"]
Jul 18 01:51:07.176669 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxfw5\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176682    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmccs\" (UniqueName: \"kubernetes.io/projected/3bb385be-cac8-481e-a541-82de95f3f191-kube-api-access-zmccs\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176794    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176853    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64a2e455-05f6-481b-98fc-20d3f9b91197-config-volume\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g8qp\" (UniqueName: \"kubernetes.io/projected/64a2e455-05f6-481b-98fc-20d3f9b91197-kube-api-access-7g8qp\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176912    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176962    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.176986    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177048    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bb385be-cac8-481e-a541-82de95f3f191-cert\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177072    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177097    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177134    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a2e455-05f6-481b-98fc-20d3f9b91197-tmp-dir\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.177258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.177156    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.277880 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.277786    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.277880 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.277846    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.277881    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.277917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bb385be-cac8-481e-a541-82de95f3f191-cert\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.278185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278113    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278151    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278202    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a2e455-05f6-481b-98fc-20d3f9b91197-tmp-dir\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.278293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278225    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxfw5\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278282    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmccs\" (UniqueName: \"kubernetes.io/projected/3bb385be-cac8-481e-a541-82de95f3f191-kube-api-access-zmccs\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278326    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278348    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278366    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278390    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64a2e455-05f6-481b-98fc-20d3f9b91197-config-volume\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g8qp\" (UniqueName: \"kubernetes.io/projected/64a2e455-05f6-481b-98fc-20d3f9b91197-kube-api-access-7g8qp\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.278491 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278816 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278646    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.278902 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278865    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.278968 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278906    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64a2e455-05f6-481b-98fc-20d3f9b91197-tmp-dir\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.279026 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.278993    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.279089 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.278227    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:07.279089 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.279040    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:07.279184 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.279109    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:07.779087303 +0000 UTC m=+37.101467464 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:07.279245 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.278832    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:07.279293 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.279285    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:07.779270621 +0000 UTC m=+37.101650768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:07.279366 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.278884    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:07.279366 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.279326    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:07.779316053 +0000 UTC m=+37.101696200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:07.280432 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.279741    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.280432 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.279765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64a2e455-05f6-481b-98fc-20d3f9b91197-config-volume\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.283618 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.283594    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.283618 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.283605    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.283618 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.283612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3bb385be-cac8-481e-a541-82de95f3f191-cert\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.287797 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.287765    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.288725 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.288700    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxfw5\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.288883 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.288862    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmccs\" (UniqueName: \"kubernetes.io/projected/3bb385be-cac8-481e-a541-82de95f3f191-kube-api-access-zmccs\") pod \"ingress-canary-5dlq8\" (UID: \"3bb385be-cac8-481e-a541-82de95f3f191\") " pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.288941 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.288925    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g8qp\" (UniqueName: \"kubernetes.io/projected/64a2e455-05f6-481b-98fc-20d3f9b91197-kube-api-access-7g8qp\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.419835 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.419798    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5dlq8"
Jul 18 01:51:07.574435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.574346    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5dlq8"]
Jul 18 01:51:07.579932 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:51:07.579901    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bb385be_cac8_481e_a541_82de95f3f191.slice/crio-f735303b3ead75ba24a417bc1e88396f9153274e4050486b8a1ff9567c99917a WatchSource:0}: Error finding container f735303b3ead75ba24a417bc1e88396f9153274e4050486b8a1ff9567c99917a: Status 404 returned error can't find the container with id f735303b3ead75ba24a417bc1e88396f9153274e4050486b8a1ff9567c99917a
Jul 18 01:51:07.782803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.782767    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:07.782803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.782803    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:07.783013 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.782946    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:07.783013 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:07.782956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:07.783013 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.782989    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:07.783161 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.783019    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:08.783001543 +0000 UTC m=+38.105381708 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:07.783161 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.783062    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:08.783033825 +0000 UTC m=+38.105413975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:07.783161 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.783075    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:07.783161 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.783089    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:07.783161 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:07.783131    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:08.783119351 +0000 UTC m=+38.105499514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:08.110243 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.110035    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:08.110409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.110035    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:08.110549 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.110035    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.113891    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.113952    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jss5k\""
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.113965    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.113972    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rbznb\""
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.113984    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:51:08.114253 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.114084    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:51:08.361874 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.361781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dlq8" event={"ID":"3bb385be-cac8-481e-a541-82de95f3f191","Type":"ContainerStarted","Data":"f735303b3ead75ba24a417bc1e88396f9153274e4050486b8a1ff9567c99917a"}
Jul 18 01:51:08.791946 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.791910    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:08.792144 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.792006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:08.792144 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:08.792034    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:08.792228 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792184    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:08.792259 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792244    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:10.792225842 +0000 UTC m=+40.114605991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:08.792685 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792666    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:08.792790 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792689    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:08.792790 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792735    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:10.792720229 +0000 UTC m=+40.115100373 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:08.792892 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792793    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:08.792892 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:08.792828    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:10.792815034 +0000 UTC m=+40.115195178 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:10.809955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:10.809859    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:10.809955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:10.809931    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:10.809955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:10.809955    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810044    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810068    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810071    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810071    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810144    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:14.810124215 +0000 UTC m=+44.132504364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810162    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:14.810154503 +0000 UTC m=+44.132534647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:10.810467 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:10.810175    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:14.810167472 +0000 UTC m=+44.132547651 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:11.372074 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:11.372039    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="4bdb976e5707639a0016ca027da75fcf65a8188b0ff474234913c8c3436521aa" exitCode=0
Jul 18 01:51:11.372263 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:11.372119    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"4bdb976e5707639a0016ca027da75fcf65a8188b0ff474234913c8c3436521aa"}
Jul 18 01:51:11.373645 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:11.373623    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5dlq8" event={"ID":"3bb385be-cac8-481e-a541-82de95f3f191","Type":"ContainerStarted","Data":"6f7ba55663618956f50edc26a50d1b102230ff81033b99aac8371b0d01886cd6"}
Jul 18 01:51:11.404714 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:11.404660    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5dlq8" podStartSLOduration=1.43676414 podStartE2EDuration="4.4046433s" totalImagesPullingTime="2.96787916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:11.40429658 +0000 UTC m=+40.726676757" watchObservedRunningTime="2026-07-18 01:51:11.4046433 +0000 UTC m=+40.727023465"
Jul 18 01:51:12.379987 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:12.379796    2560 generic.go:356] "Generic (PLEG): container finished" podID="6facbd29-cddd-4b48-80bc-76682898ab95" containerID="4f39f2efdcbfd1d58c2491bbcb999116ddf7212a308956f22176eb6b9dd98435" exitCode=0
Jul 18 01:51:12.380571 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:12.379881    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerDied","Data":"4f39f2efdcbfd1d58c2491bbcb999116ddf7212a308956f22176eb6b9dd98435"}
Jul 18 01:51:13.385743 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:13.385708    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gmx26" event={"ID":"6facbd29-cddd-4b48-80bc-76682898ab95","Type":"ContainerStarted","Data":"58909565fa76edc221e29e12b8ec96f3ba59ce192b7056aa53ca0e8118039c8e"}
Jul 18 01:51:13.410106 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:13.410049    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gmx26" podStartSLOduration=9.397793235 podStartE2EDuration="41.41002884s" totalImagesPullingTime="32.012235605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:13.408918323 +0000 UTC m=+42.731298507" watchObservedRunningTime="2026-07-18 01:51:13.41002884 +0000 UTC m=+42.732409006"
Jul 18 01:51:14.839964 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:14.839928    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:14.839964 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:14.839966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:14.840008    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840082    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840108    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840118    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840122    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840154    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:22.840135912 +0000 UTC m=+52.162516072 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840171    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:22.840163353 +0000 UTC m=+52.162543498 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:14.840409 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:14.840181    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:22.840175412 +0000 UTC m=+52.162555555 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:22.904627 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:22.904583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:22.904658    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:22.904681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904807    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904810    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904851    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904865    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:38.904847085 +0000 UTC m=+68.227227237 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904869    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904879    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:51:38.904872905 +0000 UTC m=+68.227253049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:22.905096 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:22.904922    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:51:38.904904616 +0000 UTC m=+68.227284779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:32.371347 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:32.371314    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jb5hg"
Jul 18 01:51:36.811088 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.811029    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:51:36.813412 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.813394    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 18 01:51:36.821244 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:36.821221    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 01:51:36.821297 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:36.821286    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:52:40.821267762 +0000 UTC m=+130.143647920 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : secret "metrics-daemon-secret" not found
Jul 18 01:51:36.912002 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.911962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:36.914494 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.914475    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 18 01:51:36.926490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.926457    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8ea73327-3d50-4b1a-b81e-8b49f3c91f73-original-pull-secret\") pod \"global-pull-secret-syncer-jcgp4\" (UID: \"8ea73327-3d50-4b1a-b81e-8b49f3c91f73\") " pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:36.937637 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:36.937601    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-jcgp4"
Jul 18 01:51:37.012838 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.012807    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:37.015324 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.015305    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 18 01:51:37.031266 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.031235    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 18 01:51:37.036633 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.036603    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8xld\" (UniqueName: \"kubernetes.io/projected/c3f7a9a9-a831-4d43-be7e-cc3a4b84560a-kube-api-access-d8xld\") pod \"network-check-target-fflbk\" (UID: \"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a\") " pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:37.060876 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.060841    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-jcgp4"]
Jul 18 01:51:37.225021 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.224987    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jss5k\""
Jul 18 01:51:37.233048 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.233029    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:37.367789 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.367757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fflbk"]
Jul 18 01:51:37.370691 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:51:37.370670    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3f7a9a9_a831_4d43_be7e_cc3a4b84560a.slice/crio-59ee2ad570243ce19a7e92f92ee2484bea3989da91f30023e1e2e62127231d7c WatchSource:0}: Error finding container 59ee2ad570243ce19a7e92f92ee2484bea3989da91f30023e1e2e62127231d7c: Status 404 returned error can't find the container with id 59ee2ad570243ce19a7e92f92ee2484bea3989da91f30023e1e2e62127231d7c
Jul 18 01:51:37.440714 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.440680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fflbk" event={"ID":"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a","Type":"ContainerStarted","Data":"59ee2ad570243ce19a7e92f92ee2484bea3989da91f30023e1e2e62127231d7c"}
Jul 18 01:51:37.441609 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:37.441587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jcgp4" event={"ID":"8ea73327-3d50-4b1a-b81e-8b49f3c91f73","Type":"ContainerStarted","Data":"6b18f58b7304f74f8822e1c4165d5cd153c401f46d492af7ae4e8ea20b5718f1"}
Jul 18 01:51:38.928472 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:38.928427    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:38.928550    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:38.928583    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928634    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928657    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928695    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928720    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:52:10.928701043 +0000 UTC m=+100.251081206 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928745    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:52:10.928727572 +0000 UTC m=+100.251107722 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928788    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:51:38.928939 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:51:38.928810    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:52:10.928803374 +0000 UTC m=+100.251183518 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:51:42.457671 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:42.457626    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-jcgp4" event={"ID":"8ea73327-3d50-4b1a-b81e-8b49f3c91f73","Type":"ContainerStarted","Data":"80cbd9caf86a1ed1257374a8e7c613eacbdd6b5505117d4cf595c5e65ec2647c"}
Jul 18 01:51:42.459222 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:42.459196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fflbk" event={"ID":"c3f7a9a9-a831-4d43-be7e-cc3a4b84560a","Type":"ContainerStarted","Data":"b27c465a569f9bdc4ca70263d5de21ada75c48ce8e9ced58db06542295533a29"}
Jul 18 01:51:42.459363 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:42.459348    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:51:42.472430 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:42.472377    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-jcgp4" podStartSLOduration=65.30770322 podStartE2EDuration="1m10.472355391s" totalImagesPullingTime="5.164652171s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:42.471151499 +0000 UTC m=+71.793531657" watchObservedRunningTime="2026-07-18 01:51:42.472355391 +0000 UTC m=+71.794735558"
Jul 18 01:51:42.484026 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:51:42.483926    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fflbk" podStartSLOduration=65.625865286 podStartE2EDuration="1m10.483907712s" totalImagesPullingTime="4.858042426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:51:42.483822371 +0000 UTC m=+71.806202539" watchObservedRunningTime="2026-07-18 01:51:42.483907712 +0000 UTC m=+71.806287878"
Jul 18 01:52:10.963001 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:52:10.962861    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:52:10.963001 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:52:10.962921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:52:10.963001 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:52:10.962943    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963017    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963030    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963036    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963071    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963075    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:53:14.963063019 +0000 UTC m=+164.285443162 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963147    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:53:14.963130868 +0000 UTC m=+164.285511016 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:52:10.963574 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:10.963160    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:53:14.963153801 +0000 UTC m=+164.285533945 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:52:13.464321 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:52:13.464286    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fflbk"
Jul 18 01:52:40.884895 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:52:40.884850    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:52:40.885350 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:40.884987    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 01:52:40.885350 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:52:40.885049    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:54:42.88503318 +0000 UTC m=+252.207413323 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : secret "metrics-daemon-secret" not found
Jul 18 01:53:10.085916 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:10.085857    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f"
Jul 18 01:53:10.103196 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:10.103165    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2" podUID="fc38c1c6-cf13-4fe5-b413-5d01b40954d0"
Jul 18 01:53:10.112510 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:10.112488    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tltqd" podUID="64a2e455-05f6-481b-98fc-20d3f9b91197"
Jul 18 01:53:10.673820 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:10.673781    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:53:10.674016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:10.673782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:53:10.674016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:10.673782    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:53:11.130115 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:11.130084    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:53:15.027070 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:15.027013    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:53:15.027070 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:15.027068    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:15.027121    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027160    2560 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027223    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027226    2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027240    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert podName:fc38c1c6-cf13-4fe5-b413-5d01b40954d0 nodeName:}" failed. No retries permitted until 2026-07-18 01:55:17.027221161 +0000 UTC m=+286.349601317 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert") pod "networking-console-plugin-7bc7489cb5-xjzx2" (UID: "fc38c1c6-cf13-4fe5-b413-5d01b40954d0") : secret "networking-console-plugin-cert" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027243    2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-69f6bf6697-jg9ch: secret "image-registry-tls" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027284    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls podName:a10a0f1c-7ae9-42aa-8422-751679b17d8f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:17.027269637 +0000 UTC m=+286.349649795 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls") pod "image-registry-69f6bf6697-jg9ch" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f") : secret "image-registry-tls" not found
Jul 18 01:53:15.027572 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:53:15.027306    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls podName:64a2e455-05f6-481b-98fc-20d3f9b91197 nodeName:}" failed. No retries permitted until 2026-07-18 01:55:17.02729858 +0000 UTC m=+286.349678724 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls") pod "dns-default-tltqd" (UID: "64a2e455-05f6-481b-98fc-20d3f9b91197") : secret "dns-default-metrics-tls" not found
Jul 18 01:53:25.110650 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:53:25.110550    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:54:42.942654 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:42.942598    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:54:42.943236 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:42.942758    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 18 01:54:42.943236 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:42.942852    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs podName:c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e nodeName:}" failed. No retries permitted until 2026-07-18 01:56:44.942831042 +0000 UTC m=+374.265211202 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs") pod "network-metrics-daemon-6r66r" (UID: "c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e") : secret "metrics-daemon-secret" not found
Jul 18 01:54:49.079886 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.079854    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"]
Jul 18 01:54:49.081450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.081435    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"
Jul 18 01:54:49.083610 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.083580    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"]
Jul 18 01:54:49.083842 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.083817    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-clwfq\""
Jul 18 01:54:49.085424 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.085407    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"]
Jul 18 01:54:49.085560 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.085541    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.090220 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.090085    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 18 01:54:49.090340 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.090321    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2qqsh\""
Jul 18 01:54:49.091319 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.091302    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.092410 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.092388    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:54:49.092534 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.092415    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 18 01:54:49.095710 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.095680    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:54:49.095875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.095737    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 18 01:54:49.095875 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.095789    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xvxgc\""
Jul 18 01:54:49.096107 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.096009    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 18 01:54:49.096332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.096316    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 18 01:54:49.100290 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.100246    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"]
Jul 18 01:54:49.101397 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.101376    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"]
Jul 18 01:54:49.114335 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.114311    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"]
Jul 18 01:54:49.127951 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.127917    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"]
Jul 18 01:54:49.129676 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.129660    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"]
Jul 18 01:54:49.129803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.129786    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"
Jul 18 01:54:49.131428 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.131401    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.133395 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.133377    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-529kg\""
Jul 18 01:54:49.134860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.134808    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:54:49.135969 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.135946    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-q644c\""
Jul 18 01:54:49.136062 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.135985    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 18 01:54:49.136062 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.136001    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 18 01:54:49.136062 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.136025    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 18 01:54:49.136259 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.136241    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 18 01:54:49.136928 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.136911    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79699b7d4d-l2gv2"]
Jul 18 01:54:49.138586 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.138560    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 18 01:54:49.138767 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.138753    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.141127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.141112    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 18 01:54:49.142006 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.141990    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 18 01:54:49.142092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.141993    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 18 01:54:49.142092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.142019    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 18 01:54:49.142092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.142068    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ckl6h\""
Jul 18 01:54:49.142248 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.142068    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 18 01:54:49.143024 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.143004    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 18 01:54:49.146118 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.146099    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"]
Jul 18 01:54:49.146932 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.146916    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"]
Jul 18 01:54:49.162676 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.162652    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79699b7d4d-l2gv2"]
Jul 18 01:54:49.194504 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f9jj\" (UniqueName: \"kubernetes.io/projected/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-kube-api-access-6f9jj\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.194643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194508    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.194643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194553    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdnj9\" (UniqueName: \"kubernetes.io/projected/372aca26-204b-4d70-911c-758204b8df2f-kube-api-access-sdnj9\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.194643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.194643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194605    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf9w9\" (UniqueName: \"kubernetes.io/projected/d8f740af-542b-40c4-be19-372fe4ed6873-kube-api-access-qf9w9\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/372aca26-204b-4d70-911c-758204b8df2f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-default-certificate\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8f740af-542b-40c4-be19-372fe4ed6873-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194743    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2q8q\" (UniqueName: \"kubernetes.io/projected/a8920ad0-fc6a-4ede-84d8-085c213c4ace-kube-api-access-m2q8q\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194765    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-stats-auth\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194781    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5lpb\" (UniqueName: \"kubernetes.io/projected/f9cf1546-b46c-4014-8ad9-64752a889523-kube-api-access-g5lpb\") pod \"volume-data-source-validator-78cf4c6f6-j476l\" (UID: \"f9cf1546-b46c-4014-8ad9-64752a889523\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"
Jul 18 01:54:49.194812 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194796    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.195014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194871    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8f740af-542b-40c4-be19-372fe4ed6873-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.195014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194902    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.195014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.194921    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scxgw\" (UniqueName: \"kubernetes.io/projected/b7e036ec-44be-4680-a3ef-86a464b30684-kube-api-access-scxgw\") pod \"network-check-source-7bc5b6bf85-ghql4\" (UID: \"b7e036ec-44be-4680-a3ef-86a464b30684\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"
Jul 18 01:54:49.295490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.295490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf9w9\" (UniqueName: \"kubernetes.io/projected/d8f740af-542b-40c4-be19-372fe4ed6873-kube-api-access-qf9w9\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295546    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/372aca26-204b-4d70-911c-758204b8df2f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295566    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-default-certificate\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295588    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8f740af-542b-40c4-be19-372fe4ed6873-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.295641    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295693    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2q8q\" (UniqueName: \"kubernetes.io/projected/a8920ad0-fc6a-4ede-84d8-085c213c4ace-kube-api-access-m2q8q\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.295725    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls podName:a8920ad0-fc6a-4ede-84d8-085c213c4ace nodeName:}" failed. No retries permitted until 2026-07-18 01:54:49.795706274 +0000 UTC m=+259.118086434 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-b42xk" (UID: "a8920ad0-fc6a-4ede-84d8-085c213c4ace") : secret "samples-operator-tls" not found
Jul 18 01:54:49.295752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295750    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-stats-auth\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295787    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5lpb\" (UniqueName: \"kubernetes.io/projected/f9cf1546-b46c-4014-8ad9-64752a889523-kube-api-access-g5lpb\") pod \"volume-data-source-validator-78cf4c6f6-j476l\" (UID: \"f9cf1546-b46c-4014-8ad9-64752a889523\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295811    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8f740af-542b-40c4-be19-372fe4ed6873-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295895    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295922    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scxgw\" (UniqueName: \"kubernetes.io/projected/b7e036ec-44be-4680-a3ef-86a464b30684-kube-api-access-scxgw\") pod \"network-check-source-7bc5b6bf85-ghql4\" (UID: \"b7e036ec-44be-4680-a3ef-86a464b30684\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.295968    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f9jj\" (UniqueName: \"kubernetes.io/projected/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-kube-api-access-6f9jj\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.296009    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.296121 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.296040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdnj9\" (UniqueName: \"kubernetes.io/projected/372aca26-204b-4d70-911c-758204b8df2f-kube-api-access-sdnj9\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.296310    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:49.796291739 +0000 UTC m=+259.118671884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : configmap references non-existent config key: service-ca.crt
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.296341    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.296382    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.296385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/372aca26-204b-4d70-911c-758204b8df2f-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.296388    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d8f740af-542b-40c4-be19-372fe4ed6873-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.296424    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls podName:372aca26-204b-4d70-911c-758204b8df2f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:49.796411159 +0000 UTC m=+259.118791307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-bbl66" (UID: "372aca26-204b-4d70-911c-758204b8df2f") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:49.296492 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.296459    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:49.796442691 +0000 UTC m=+259.118822849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : secret "router-metrics-certs-default" not found
Jul 18 01:54:49.298435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.298406    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-stats-auth\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.298606 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.298403    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-default-certificate\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.298606 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.298480    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d8f740af-542b-40c4-be19-372fe4ed6873-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.305133 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.305102    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2q8q\" (UniqueName: \"kubernetes.io/projected/a8920ad0-fc6a-4ede-84d8-085c213c4ace-kube-api-access-m2q8q\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.306497 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.306474    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f9jj\" (UniqueName: \"kubernetes.io/projected/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-kube-api-access-6f9jj\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.306635 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.306601    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdnj9\" (UniqueName: \"kubernetes.io/projected/372aca26-204b-4d70-911c-758204b8df2f-kube-api-access-sdnj9\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.306983 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.306962    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5lpb\" (UniqueName: \"kubernetes.io/projected/f9cf1546-b46c-4014-8ad9-64752a889523-kube-api-access-g5lpb\") pod \"volume-data-source-validator-78cf4c6f6-j476l\" (UID: \"f9cf1546-b46c-4014-8ad9-64752a889523\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"
Jul 18 01:54:49.307065 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.306987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scxgw\" (UniqueName: \"kubernetes.io/projected/b7e036ec-44be-4680-a3ef-86a464b30684-kube-api-access-scxgw\") pod \"network-check-source-7bc5b6bf85-ghql4\" (UID: \"b7e036ec-44be-4680-a3ef-86a464b30684\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"
Jul 18 01:54:49.307270 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.307249    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf9w9\" (UniqueName: \"kubernetes.io/projected/d8f740af-542b-40c4-be19-372fe4ed6873-kube-api-access-qf9w9\") pod \"kube-storage-version-migrator-operator-7658d9b749-t4m44\" (UID: \"d8f740af-542b-40c4-be19-372fe4ed6873\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.393643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.393612    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"
Jul 18 01:54:49.405353 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.405325    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"
Jul 18 01:54:49.441531 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.440170    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"
Jul 18 01:54:49.545690 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.545653    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4"]
Jul 18 01:54:49.548969 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:54:49.548936    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7e036ec_44be_4680_a3ef_86a464b30684.slice/crio-6a43c6967d3e5dbf8a1e13d55eb135d48cd61c75210f19e9a33075a2e2d0fa69 WatchSource:0}: Error finding container 6a43c6967d3e5dbf8a1e13d55eb135d48cd61c75210f19e9a33075a2e2d0fa69: Status 404 returned error can't find the container with id 6a43c6967d3e5dbf8a1e13d55eb135d48cd61c75210f19e9a33075a2e2d0fa69
Jul 18 01:54:49.570021 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.569876    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44"]
Jul 18 01:54:49.573165 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:54:49.573131    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8f740af_542b_40c4_be19_372fe4ed6873.slice/crio-6be762f8fdd7cf85e306dc92385c642b53058364d5f988d452367544cbfbaff9 WatchSource:0}: Error finding container 6be762f8fdd7cf85e306dc92385c642b53058364d5f988d452367544cbfbaff9: Status 404 returned error can't find the container with id 6be762f8fdd7cf85e306dc92385c642b53058364d5f988d452367544cbfbaff9
Jul 18 01:54:49.587302 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.587277    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l"]
Jul 18 01:54:49.590434 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:54:49.590405    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9cf1546_b46c_4014_8ad9_64752a889523.slice/crio-7997e49cd2fc392af48fea8c1d4b804ef6469cc8daa400c9f00ba9946da9b649 WatchSource:0}: Error finding container 7997e49cd2fc392af48fea8c1d4b804ef6469cc8daa400c9f00ba9946da9b649: Status 404 returned error can't find the container with id 7997e49cd2fc392af48fea8c1d4b804ef6469cc8daa400c9f00ba9946da9b649
Jul 18 01:54:49.801655 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.801568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:49.801655 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.801612    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:49.801876 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801732    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:54:49.801876 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801742    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:49.801876 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801779    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls podName:a8920ad0-fc6a-4ede-84d8-085c213c4ace nodeName:}" failed. No retries permitted until 2026-07-18 01:54:50.801764276 +0000 UTC m=+260.124144422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-b42xk" (UID: "a8920ad0-fc6a-4ede-84d8-085c213c4ace") : secret "samples-operator-tls" not found
Jul 18 01:54:49.801876 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.801814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.801876 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801845    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:54:49.802082 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801885    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:50.801868074 +0000 UTC m=+260.124248235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : secret "router-metrics-certs-default" not found
Jul 18 01:54:49.802082 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.801926    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls podName:372aca26-204b-4d70-911c-758204b8df2f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:50.801915387 +0000 UTC m=+260.124295534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-bbl66" (UID: "372aca26-204b-4d70-911c-758204b8df2f") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:49.802082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.801946    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:49.802182 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:49.802099    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:50.802081604 +0000 UTC m=+260.124461763 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : configmap references non-existent config key: service-ca.crt
Jul 18 01:54:49.899452 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.899420    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l" event={"ID":"f9cf1546-b46c-4014-8ad9-64752a889523","Type":"ContainerStarted","Data":"7997e49cd2fc392af48fea8c1d4b804ef6469cc8daa400c9f00ba9946da9b649"}
Jul 18 01:54:49.900509 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.900482    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44" event={"ID":"d8f740af-542b-40c4-be19-372fe4ed6873","Type":"ContainerStarted","Data":"6be762f8fdd7cf85e306dc92385c642b53058364d5f988d452367544cbfbaff9"}
Jul 18 01:54:49.901978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.901957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4" event={"ID":"b7e036ec-44be-4680-a3ef-86a464b30684","Type":"ContainerStarted","Data":"d9b265c8edda72250d11867ad197a999bb4c0a413f3467736afafb2f52c4179f"}
Jul 18 01:54:49.902083 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.901992    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4" event={"ID":"b7e036ec-44be-4680-a3ef-86a464b30684","Type":"ContainerStarted","Data":"6a43c6967d3e5dbf8a1e13d55eb135d48cd61c75210f19e9a33075a2e2d0fa69"}
Jul 18 01:54:49.920008 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:49.919958    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-ghql4" podStartSLOduration=0.919942233 podStartE2EDuration="919.942233ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:49.918695141 +0000 UTC m=+259.241075308" watchObservedRunningTime="2026-07-18 01:54:49.919942233 +0000 UTC m=+259.242322399"
Jul 18 01:54:50.810409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.810382    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.810468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.810531    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810549    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.810585    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810609    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls podName:a8920ad0-fc6a-4ede-84d8-085c213c4ace nodeName:}" failed. No retries permitted until 2026-07-18 01:54:52.810591371 +0000 UTC m=+262.132971522 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-b42xk" (UID: "a8920ad0-fc6a-4ede-84d8-085c213c4ace") : secret "samples-operator-tls" not found
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810623    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810651    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810682    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:52.810665106 +0000 UTC m=+262.133045250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : secret "router-metrics-certs-default" not found
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810696    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:52.810690686 +0000 UTC m=+262.133070830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : configmap references non-existent config key: service-ca.crt
Jul 18 01:54:50.810816 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:50.810706    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls podName:372aca26-204b-4d70-911c-758204b8df2f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:52.810701225 +0000 UTC m=+262.133081369 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-bbl66" (UID: "372aca26-204b-4d70-911c-758204b8df2f") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:50.906534 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.906468    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l" event={"ID":"f9cf1546-b46c-4014-8ad9-64752a889523","Type":"ContainerStarted","Data":"5b1bf19db9b46385c73d0d60ae4aa8be9454fef7606c98e9e198b58f9179bb7b"}
Jul 18 01:54:50.922203 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:50.922149    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-j476l" podStartSLOduration=0.751906869 podStartE2EDuration="1.92213064s" totalImagesPullingTime="1.170223771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:50.921238597 +0000 UTC m=+260.243618764" watchObservedRunningTime="2026-07-18 01:54:50.92213064 +0000 UTC m=+260.244510783"
Jul 18 01:54:51.915291 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:51.915252    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44" event={"ID":"d8f740af-542b-40c4-be19-372fe4ed6873","Type":"ContainerStarted","Data":"e0d2935a9f7c1cdd772688bf7927641875ad7d28b61a7bd47fe21dfb0e888448"}
Jul 18 01:54:51.947429 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:51.947370    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44" podStartSLOduration=1.140191041 podStartE2EDuration="2.947353496s" totalImagesPullingTime="1.807162455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:54:51.946788651 +0000 UTC m=+261.269168818" watchObservedRunningTime="2026-07-18 01:54:51.947353496 +0000 UTC m=+261.269733704"
Jul 18 01:54:52.826601 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:52.826547    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:52.826774 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:52.826633    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:52.826774 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:52.826675    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:52.826774 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826701    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:54:52.826774 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826767    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:56.826751447 +0000 UTC m=+266.149131594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : secret "router-metrics-certs-default" not found
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826782    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826783    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826798    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:54:56.826779041 +0000 UTC m=+266.149159190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : configmap references non-existent config key: service-ca.crt
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826824    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls podName:a8920ad0-fc6a-4ede-84d8-085c213c4ace nodeName:}" failed. No retries permitted until 2026-07-18 01:54:56.82681452 +0000 UTC m=+266.149194667 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-b42xk" (UID: "a8920ad0-fc6a-4ede-84d8-085c213c4ace") : secret "samples-operator-tls" not found
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:52.826709    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:52.826948 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:52.826835    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls podName:372aca26-204b-4d70-911c-758204b8df2f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:56.826830215 +0000 UTC m=+266.149210359 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-bbl66" (UID: "372aca26-204b-4d70-911c-758204b8df2f") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:53.918941 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.918900    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cmp2v"]
Jul 18 01:54:53.922285 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.922265    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:53.926379 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.926353    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 18 01:54:53.938537 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.938500    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 18 01:54:53.939575 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.939554    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 18 01:54:53.939704 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.939635    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 18 01:54:53.939704 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.939635    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qnqgm\""
Jul 18 01:54:53.943063 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:53.943039    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmp2v"]
Jul 18 01:54:54.037631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.037594    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2zjn\" (UniqueName: \"kubernetes.io/projected/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-api-access-f2zjn\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.037631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.037633    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.037821 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.037659    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-crio-socket\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.037821 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.037762    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.037821 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.037804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-data-volume\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.138895 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.138838    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.138915    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-crio-socket\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.138963    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139081 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.138971    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:54:54.139081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.138999    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-data-volume\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139081 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.139038    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:54.639019541 +0000 UTC m=+263.961399707 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : configmap "kube-rbac-proxy" not found
Jul 18 01:54:54.139328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.139070    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-crio-socket\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139328 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.139104    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:54.139328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.139146    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2zjn\" (UniqueName: \"kubernetes.io/projected/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-api-access-f2zjn\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.139328 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.139194    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:54.63918172 +0000 UTC m=+263.961561864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:54.139328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.139298    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-data-volume\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.148004 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.147968    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2zjn\" (UniqueName: \"kubernetes.io/projected/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-api-access-f2zjn\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.644027 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.643985    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.644247 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:54.644052    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:54.644247 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.644138    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:54:54.644247 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.644179    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:54.644247 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.644220    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:55.644199075 +0000 UTC m=+264.966579239 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : configmap "kube-rbac-proxy" not found
Jul 18 01:54:54.644247 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:54.644239    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:55.644231269 +0000 UTC m=+264.966611413 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:55.655005 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:55.654966    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:55.655444 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:55.655046    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:55.655444 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:55.655098    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:54:55.655444 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:55.655178    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:57.655155995 +0000 UTC m=+266.977536139 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : configmap "kube-rbac-proxy" not found
Jul 18 01:54:55.655444 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:55.655191    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:55.655444 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:55.655235    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:54:57.655222857 +0000 UTC m=+266.977603004 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:56.865838 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:56.865792    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:56.865857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:56.865887    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:56.865959    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866031    2560 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866058    2560 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866041    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:55:04.866014486 +0000 UTC m=+274.188394650 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : configmap references non-existent config key: service-ca.crt
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866086    2560 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866106    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls podName:372aca26-204b-4d70-911c-758204b8df2f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:04.866087538 +0000 UTC m=+274.188467691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-bbl66" (UID: "372aca26-204b-4d70-911c-758204b8df2f") : secret "cluster-monitoring-operator-tls" not found
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866130    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs podName:e064ccab-2a4b-4ad3-a893-224c1cc5eca0 nodeName:}" failed. No retries permitted until 2026-07-18 01:55:04.866120423 +0000 UTC m=+274.188500567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs") pod "router-default-79699b7d4d-l2gv2" (UID: "e064ccab-2a4b-4ad3-a893-224c1cc5eca0") : secret "router-metrics-certs-default" not found
Jul 18 01:54:56.866315 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:56.866143    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls podName:a8920ad0-fc6a-4ede-84d8-085c213c4ace nodeName:}" failed. No retries permitted until 2026-07-18 01:55:04.866137496 +0000 UTC m=+274.188517640 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-b42xk" (UID: "a8920ad0-fc6a-4ede-84d8-085c213c4ace") : secret "samples-operator-tls" not found
Jul 18 01:54:56.884272 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:56.884242    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qcblm_93de963b-7c22-4e91-a404-82baa95cafc7/dns-node-resolver/0.log"
Jul 18 01:54:57.472137 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:57.472113    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q7wjb_ac9788af-d9b4-4af3-8220-d2b97fdc56d0/node-ca/0.log"
Jul 18 01:54:57.673452 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:57.673410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:57.673636 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:57.673476    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:54:57.673636 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:57.673568    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:54:57.673700 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:57.673640    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:57.673700 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:57.673646    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:01.673625571 +0000 UTC m=+270.996005734 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : configmap "kube-rbac-proxy" not found
Jul 18 01:54:57.673700 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:54:57.673685    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:01.673675259 +0000 UTC m=+270.996055407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:54:58.272924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:58.272900    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dlq8_3bb385be-cac8-481e-a541-82de95f3f191/serve-healthcheck-canary/0.log"
Jul 18 01:54:59.267233 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:54:59.267206    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-t4m44_d8f740af-542b-40c4-be19-372fe4ed6873/kube-storage-version-migrator-operator/0.log"
Jul 18 01:55:01.706561 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:01.706497    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:01.706964 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:01.706606    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:01.706964 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:01.706651    2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 18 01:55:01.706964 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:01.706729    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:09.706711792 +0000 UTC m=+279.029091939 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : configmap "kube-rbac-proxy" not found
Jul 18 01:55:01.706964 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:01.706774    2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 18 01:55:01.706964 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:01.706842    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls podName:4a8317e4-bba8-4b86-80b0-f1ea0208f85f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:09.706828562 +0000 UTC m=+279.029208707 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cmp2v" (UID: "4a8317e4-bba8-4b86-80b0-f1ea0208f85f") : secret "insights-runtime-extractor-tls" not found
Jul 18 01:55:04.935482 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.935443    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:04.936014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.935544    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:04.936014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.935600    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:55:04.936014 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.935632    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:55:04.936254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.936229    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-service-ca-bundle\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:04.938169 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.938134    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a8920ad0-fc6a-4ede-84d8-085c213c4ace-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-b42xk\" (UID: \"a8920ad0-fc6a-4ede-84d8-085c213c4ace\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:55:04.938289 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.938214    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/372aca26-204b-4d70-911c-758204b8df2f-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-bbl66\" (UID: \"372aca26-204b-4d70-911c-758204b8df2f\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:55:04.938332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.938300    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e064ccab-2a4b-4ad3-a893-224c1cc5eca0-metrics-certs\") pod \"router-default-79699b7d4d-l2gv2\" (UID: \"e064ccab-2a4b-4ad3-a893-224c1cc5eca0\") " pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:04.999361 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:04.999322    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"
Jul 18 01:55:05.046788 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.046745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"
Jul 18 01:55:05.051580 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.051552    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:05.147181 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.147148    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk"]
Jul 18 01:55:05.194699 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.194626    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66"]
Jul 18 01:55:05.198233 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:05.198197    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372aca26_204b_4d70_911c_758204b8df2f.slice/crio-718b0f1f0c9bab5c8ba3bc650c14bd0372266e4ed83896d7aa12505c99680616 WatchSource:0}: Error finding container 718b0f1f0c9bab5c8ba3bc650c14bd0372266e4ed83896d7aa12505c99680616: Status 404 returned error can't find the container with id 718b0f1f0c9bab5c8ba3bc650c14bd0372266e4ed83896d7aa12505c99680616
Jul 18 01:55:05.215958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.215932    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79699b7d4d-l2gv2"]
Jul 18 01:55:05.218563 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:05.218539    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode064ccab_2a4b_4ad3_a893_224c1cc5eca0.slice/crio-28b00828963b77f0fa1f3f49eae94c656a7d5fe18cb269c44ae52a81be05e5c5 WatchSource:0}: Error finding container 28b00828963b77f0fa1f3f49eae94c656a7d5fe18cb269c44ae52a81be05e5c5: Status 404 returned error can't find the container with id 28b00828963b77f0fa1f3f49eae94c656a7d5fe18cb269c44ae52a81be05e5c5
Jul 18 01:55:05.959281 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.959219    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk" event={"ID":"a8920ad0-fc6a-4ede-84d8-085c213c4ace","Type":"ContainerStarted","Data":"55b8e5df9a9c6024ab5ab704a5702edcdccd245b4e76194791e9c231d03e29bf"}
Jul 18 01:55:05.961008 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.960971    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79699b7d4d-l2gv2" event={"ID":"e064ccab-2a4b-4ad3-a893-224c1cc5eca0","Type":"ContainerStarted","Data":"96064da4a4898b45de7c1c77f94ed969f13b7a58ac1b5fdeddb23f86ed4ac5fb"}
Jul 18 01:55:05.961138 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.961030    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79699b7d4d-l2gv2" event={"ID":"e064ccab-2a4b-4ad3-a893-224c1cc5eca0","Type":"ContainerStarted","Data":"28b00828963b77f0fa1f3f49eae94c656a7d5fe18cb269c44ae52a81be05e5c5"}
Jul 18 01:55:05.962815 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:05.962781    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66" event={"ID":"372aca26-204b-4d70-911c-758204b8df2f","Type":"ContainerStarted","Data":"718b0f1f0c9bab5c8ba3bc650c14bd0372266e4ed83896d7aa12505c99680616"}
Jul 18 01:55:06.011180 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:06.010789    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79699b7d4d-l2gv2" podStartSLOduration=17.010767403 podStartE2EDuration="17.010767403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:06.00981379 +0000 UTC m=+275.332193967" watchObservedRunningTime="2026-07-18 01:55:06.010767403 +0000 UTC m=+275.333147547"
Jul 18 01:55:06.051824 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:06.051785    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:06.054777 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:06.054752    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:06.966179 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:06.966132    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:06.967651 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:06.967629    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79699b7d4d-l2gv2"
Jul 18 01:55:07.973232 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:07.973196    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk" event={"ID":"a8920ad0-fc6a-4ede-84d8-085c213c4ace","Type":"ContainerStarted","Data":"f5ecf1227f4ca495aaaadde1a015c9c3b64e4ce2619fedff536f14517b7768cd"}
Jul 18 01:55:07.973711 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:07.973260    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk" event={"ID":"a8920ad0-fc6a-4ede-84d8-085c213c4ace","Type":"ContainerStarted","Data":"c9385a42a5ece78213f01f669e8f238863b46c8e16f5fe7a4784204e4b87a0a1"}
Jul 18 01:55:07.974725 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:07.974699    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66" event={"ID":"372aca26-204b-4d70-911c-758204b8df2f","Type":"ContainerStarted","Data":"377c80cbd52619b362c49ba0a39c9ce67d67bb1ffce8949d3681b104ca45bad0"}
Jul 18 01:55:07.989424 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:07.989369    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-b42xk" podStartSLOduration=17.010289239 podStartE2EDuration="18.989350112s" totalImagesPullingTime="1.979060873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:07.988860636 +0000 UTC m=+277.311240801" watchObservedRunningTime="2026-07-18 01:55:07.989350112 +0000 UTC m=+277.311730278"
Jul 18 01:55:08.003278 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:08.003230    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-bbl66" podStartSLOduration=17.013979203 podStartE2EDuration="19.003215069s" totalImagesPullingTime="1.989235866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:08.002947518 +0000 UTC m=+277.325327686" watchObservedRunningTime="2026-07-18 01:55:08.003215069 +0000 UTC m=+277.325595234"
Jul 18 01:55:09.783368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.783331    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:09.783780 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.783393    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:09.783876 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.783858    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:09.785836 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.785816    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4a8317e4-bba8-4b86-80b0-f1ea0208f85f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cmp2v\" (UID: \"4a8317e4-bba8-4b86-80b0-f1ea0208f85f\") " pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:09.830684 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.830647    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cmp2v"
Jul 18 01:55:09.965237 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.965155    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cmp2v"]
Jul 18 01:55:09.967504 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:09.967481    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8317e4_bba8_4b86_80b0_f1ea0208f85f.slice/crio-10592135cda02968b9fdf9fbef9da5f57adbc69c8325f9417a941203af9a92e1 WatchSource:0}: Error finding container 10592135cda02968b9fdf9fbef9da5f57adbc69c8325f9417a941203af9a92e1: Status 404 returned error can't find the container with id 10592135cda02968b9fdf9fbef9da5f57adbc69c8325f9417a941203af9a92e1
Jul 18 01:55:09.980716 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:09.980680    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmp2v" event={"ID":"4a8317e4-bba8-4b86-80b0-f1ea0208f85f","Type":"ContainerStarted","Data":"10592135cda02968b9fdf9fbef9da5f57adbc69c8325f9417a941203af9a92e1"}
Jul 18 01:55:13.675021 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:13.674977    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2" podUID="fc38c1c6-cf13-4fe5-b413-5d01b40954d0"
Jul 18 01:55:13.675021 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:13.674982    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tltqd" podUID="64a2e455-05f6-481b-98fc-20d3f9b91197"
Jul 18 01:55:13.675476 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:13.674982    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f"
Jul 18 01:55:13.995257 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:13.995173    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:55:13.995257 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:13.995192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:13.995612 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:13.995312    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:15.000436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:15.000396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmp2v" event={"ID":"4a8317e4-bba8-4b86-80b0-f1ea0208f85f","Type":"ContainerStarted","Data":"87254ae20486f5d1e27c6cc382cd1a558b8fcb29ec6ec13ae45f2e7b3d2b0073"}
Jul 18 01:55:17.048402 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.048350    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:17.048902 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.048455    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:55:17.048902 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.048487    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:17.051322 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.051289    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"image-registry-69f6bf6697-jg9ch\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") " pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:17.051571 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.051543    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64a2e455-05f6-481b-98fc-20d3f9b91197-metrics-tls\") pod \"dns-default-tltqd\" (UID: \"64a2e455-05f6-481b-98fc-20d3f9b91197\") " pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:17.051690 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.051612    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/fc38c1c6-cf13-4fe5-b413-5d01b40954d0-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-xjzx2\" (UID: \"fc38c1c6-cf13-4fe5-b413-5d01b40954d0\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:55:17.301099 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.301004    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qqc6n\""
Jul 18 01:55:17.301099 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.301043    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-dwp4g\""
Jul 18 01:55:17.301311 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.301021    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2nvr5\""
Jul 18 01:55:17.306806 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.306768    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:17.306806 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.306784    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:17.307010 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:17.306791    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"
Jul 18 01:55:18.428193 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:18.428165    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tltqd"]
Jul 18 01:55:18.431890 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:18.431856    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a2e455_05f6_481b_98fc_20d3f9b91197.slice/crio-a627414ef3b7e7f14b1489a95f227e814bc6c24bcb3cb4d0f810dc31144f63cc WatchSource:0}: Error finding container a627414ef3b7e7f14b1489a95f227e814bc6c24bcb3cb4d0f810dc31144f63cc: Status 404 returned error can't find the container with id a627414ef3b7e7f14b1489a95f227e814bc6c24bcb3cb4d0f810dc31144f63cc
Jul 18 01:55:18.438783 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:18.438755    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2"]
Jul 18 01:55:18.442055 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:18.442025    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc38c1c6_cf13_4fe5_b413_5d01b40954d0.slice/crio-14f3f87d600399d21cfb4870ed2927758177e5c49f543c3f81c382f1f47565f6 WatchSource:0}: Error finding container 14f3f87d600399d21cfb4870ed2927758177e5c49f543c3f81c382f1f47565f6: Status 404 returned error can't find the container with id 14f3f87d600399d21cfb4870ed2927758177e5c49f543c3f81c382f1f47565f6
Jul 18 01:55:18.464987 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:18.464954    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:55:18.467976 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:18.467950    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda10a0f1c_7ae9_42aa_8422_751679b17d8f.slice/crio-cd74f01f45006321244fecae6f7c1301eda332214feffb2922c342b03558de80 WatchSource:0}: Error finding container cd74f01f45006321244fecae6f7c1301eda332214feffb2922c342b03558de80: Status 404 returned error can't find the container with id cd74f01f45006321244fecae6f7c1301eda332214feffb2922c342b03558de80
Jul 18 01:55:19.019614 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.019579    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" event={"ID":"a10a0f1c-7ae9-42aa-8422-751679b17d8f","Type":"ContainerStarted","Data":"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"}
Jul 18 01:55:19.019858 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.019643    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" event={"ID":"a10a0f1c-7ae9-42aa-8422-751679b17d8f","Type":"ContainerStarted","Data":"cd74f01f45006321244fecae6f7c1301eda332214feffb2922c342b03558de80"}
Jul 18 01:55:19.020468 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.020444    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:19.022632 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.022602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmp2v" event={"ID":"4a8317e4-bba8-4b86-80b0-f1ea0208f85f","Type":"ContainerStarted","Data":"8885d77db8387ee45b26d0ba0f24b16dfaf082d3fd5f48104433417af2fbb9e8"}
Jul 18 01:55:19.023914 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.023884    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2" event={"ID":"fc38c1c6-cf13-4fe5-b413-5d01b40954d0","Type":"ContainerStarted","Data":"14f3f87d600399d21cfb4870ed2927758177e5c49f543c3f81c382f1f47565f6"}
Jul 18 01:55:19.025434 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.025409    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tltqd" event={"ID":"64a2e455-05f6-481b-98fc-20d3f9b91197","Type":"ContainerStarted","Data":"a627414ef3b7e7f14b1489a95f227e814bc6c24bcb3cb4d0f810dc31144f63cc"}
Jul 18 01:55:19.043347 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:19.043283    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" podStartSLOduration=287.04325968 podStartE2EDuration="4m47.04325968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:19.043175741 +0000 UTC m=+288.365555908" watchObservedRunningTime="2026-07-18 01:55:19.04325968 +0000 UTC m=+288.365639847"
Jul 18 01:55:21.036818 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:21.036780    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2" event={"ID":"fc38c1c6-cf13-4fe5-b413-5d01b40954d0","Type":"ContainerStarted","Data":"7bd793b6e0a1c30bc3a4e4e731ad0aae67498557e8d3f4d3b7fdaf63c75b94aa"}
Jul 18 01:55:21.038586 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:21.038559    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tltqd" event={"ID":"64a2e455-05f6-481b-98fc-20d3f9b91197","Type":"ContainerStarted","Data":"0b8c65f29bcde5ca227b4e1163ba26f538d599204ba0342b9eac2947e490bd1a"}
Jul 18 01:55:21.054339 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:21.054277    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-xjzx2" podStartSLOduration=275.230641 podStartE2EDuration="4m37.054251165s" totalImagesPullingTime="1.823610165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:21.053616882 +0000 UTC m=+290.375997060" watchObservedRunningTime="2026-07-18 01:55:21.054251165 +0000 UTC m=+290.376631332"
Jul 18 01:55:23.047406 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.047354    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cmp2v" event={"ID":"4a8317e4-bba8-4b86-80b0-f1ea0208f85f","Type":"ContainerStarted","Data":"4e65dc47d741a2ece7a182ad0ccd61af542d3adbdf864226aef37ad99c873494"}
Jul 18 01:55:23.049111 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.049086    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tltqd" event={"ID":"64a2e455-05f6-481b-98fc-20d3f9b91197","Type":"ContainerStarted","Data":"995f7826ebe1fb387c93fd3d947fc0a7e4e03266c27c52635c1aec323157492f"}
Jul 18 01:55:23.049248 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.049233    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:23.063983 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.063935    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cmp2v" podStartSLOduration=18.002695458 podStartE2EDuration="30.063921401s" totalImagesPullingTime="12.061225943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 01:54:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:23.062500872 +0000 UTC m=+292.384881038" watchObservedRunningTime="2026-07-18 01:55:23.063921401 +0000 UTC m=+292.386301600"
Jul 18 01:55:23.079907 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.079857    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tltqd" podStartSLOduration=254.201339848 podStartE2EDuration="4m16.079837754s" totalImagesPullingTime="1.878497906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:51:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:23.078539877 +0000 UTC m=+292.400920042" watchObservedRunningTime="2026-07-18 01:55:23.079837754 +0000 UTC m=+292.402217919"
Jul 18 01:55:23.734806 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.734770    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"]
Jul 18 01:55:23.738957 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.738940    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.741427 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.741407    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 18 01:55:23.742558 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.742541    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bzm9j\""
Jul 18 01:55:23.742662 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.742625    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 18 01:55:23.742722 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.742657    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 18 01:55:23.748061 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.748037    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"]
Jul 18 01:55:23.805387 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.805344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.805387 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.805384    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.805615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.805456    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k57tt\" (UniqueName: \"kubernetes.io/projected/fc902078-3551-4bfe-b516-922b169b69f4-kube-api-access-k57tt\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.805615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.805564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc902078-3551-4bfe-b516-922b169b69f4-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.906128 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.906087    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k57tt\" (UniqueName: \"kubernetes.io/projected/fc902078-3551-4bfe-b516-922b169b69f4-kube-api-access-k57tt\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.906293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.906193    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc902078-3551-4bfe-b516-922b169b69f4-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.906293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.906226    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.906293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.906245    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.906851 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.906827    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc902078-3551-4bfe-b516-922b169b69f4-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.908727 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.908699    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.908925 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.908905    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fc902078-3551-4bfe-b516-922b169b69f4-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:23.913543 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:23.913504    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k57tt\" (UniqueName: \"kubernetes.io/projected/fc902078-3551-4bfe-b516-922b169b69f4-kube-api-access-k57tt\") pod \"prometheus-operator-5945b5cfd-xkvkl\" (UID: \"fc902078-3551-4bfe-b516-922b169b69f4\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:24.048039 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:24.047954    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"
Jul 18 01:55:24.173974 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:24.173946    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl"]
Jul 18 01:55:24.176297 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:24.176263    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc902078_3551_4bfe_b516_922b169b69f4.slice/crio-9dcd9426192b2c7077c197662b307f00fa7d136a8b609ef637175467df73703d WatchSource:0}: Error finding container 9dcd9426192b2c7077c197662b307f00fa7d136a8b609ef637175467df73703d: Status 404 returned error can't find the container with id 9dcd9426192b2c7077c197662b307f00fa7d136a8b609ef637175467df73703d
Jul 18 01:55:25.057410 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.057378    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl" event={"ID":"fc902078-3551-4bfe-b516-922b169b69f4","Type":"ContainerStarted","Data":"9dcd9426192b2c7077c197662b307f00fa7d136a8b609ef637175467df73703d"}
Jul 18 01:55:25.760706 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.760674    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:55:25.764849 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.764832    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.767406 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.767360    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 18 01:55:25.767406 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.767389    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 18 01:55:25.767406 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.767397    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 18 01:55:25.768871 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.768850    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 18 01:55:25.769000 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.768861    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 18 01:55:25.769000 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.768978    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 18 01:55:25.769086 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.768934    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 18 01:55:25.769086 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.768937    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-t4dvc\""
Jul 18 01:55:25.774307 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.774285    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:55:25.818070 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.818070 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818070    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.818275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818099    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddc7x\" (UniqueName: \"kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.818275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818166    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.818275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818213    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.818275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.818243    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918579    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918763 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918623    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918763 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918648    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918763 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918884 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.918884 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.918814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddc7x\" (UniqueName: \"kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.919283 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.919250    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.919383 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.919313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.919383 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.919342    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.921341 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.921324    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.921400 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.921321    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:25.926757 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:25.926723    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddc7x\" (UniqueName: \"kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x\") pod \"console-b669c55b-xvrhm\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") " pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:26.062946 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:26.062861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl" event={"ID":"fc902078-3551-4bfe-b516-922b169b69f4","Type":"ContainerStarted","Data":"d59e1044330677aa319539ded75f53a6fb27d840a36cfd04180119ba457860b9"}
Jul 18 01:55:26.062946 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:26.062914    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl" event={"ID":"fc902078-3551-4bfe-b516-922b169b69f4","Type":"ContainerStarted","Data":"8618a4657b11eb9555deede9c6a3b64bf5f41fc5d13174e04a73915ea43689a1"}
Jul 18 01:55:26.074066 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:26.074038    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:26.078036 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:26.077990    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-xkvkl" podStartSLOduration=1.947858236 podStartE2EDuration="3.077974793s" totalImagesPullingTime="1.130116557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:26.07736733 +0000 UTC m=+295.399747526" watchObservedRunningTime="2026-07-18 01:55:26.077974793 +0000 UTC m=+295.400354959"
Jul 18 01:55:26.204411 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:26.204373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:55:26.207444 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:26.207421    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00c655a_c222_4203_b371_5da034253e34.slice/crio-dfbd8c6d3b5c110b9f62318cad72f6cfeb0e97c3b8404655087e2dc5d32ac014 WatchSource:0}: Error finding container dfbd8c6d3b5c110b9f62318cad72f6cfeb0e97c3b8404655087e2dc5d32ac014: Status 404 returned error can't find the container with id dfbd8c6d3b5c110b9f62318cad72f6cfeb0e97c3b8404655087e2dc5d32ac014
Jul 18 01:55:27.068437 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:27.068398    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b669c55b-xvrhm" event={"ID":"a00c655a-c222-4203-b371-5da034253e34","Type":"ContainerStarted","Data":"dfbd8c6d3b5c110b9f62318cad72f6cfeb0e97c3b8404655087e2dc5d32ac014"}
Jul 18 01:55:28.111536 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:28.111471    2560 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-6r66r" podUID="c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e"
Jul 18 01:55:28.136503 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.136461    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-spmxg"]
Jul 18 01:55:28.141490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.141459    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.144435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.144408    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 18 01:55:28.144643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.144489    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2cqq9\""
Jul 18 01:55:28.145317 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.145270    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 18 01:55:28.145317 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.145295    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 18 01:55:28.237344 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-root\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-sys\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237392    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-accelerators-collector-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-textfile\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237472    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-wtmp\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237571    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237625    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-metrics-client-ca\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237651    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-tls\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.237742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.237673    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp9jd\" (UniqueName: \"kubernetes.io/projected/b429c098-3403-4f95-9052-fe7a8b995170-kube-api-access-fp9jd\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338097 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338062    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-sys\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338097 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338099    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-accelerators-collector-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338148    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-textfile\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338179    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-wtmp\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338208    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-sys\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338218    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-metrics-client-ca\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338347    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-tls\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338376    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp9jd\" (UniqueName: \"kubernetes.io/projected/b429c098-3403-4f95-9052-fe7a8b995170-kube-api-access-fp9jd\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338407    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-root\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338810 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338413    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-wtmp\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338810 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338472    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b429c098-3403-4f95-9052-fe7a8b995170-root\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338810 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-textfile\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.338954 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.338848    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-accelerators-collector-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.339192 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.339148    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b429c098-3403-4f95-9052-fe7a8b995170-metrics-client-ca\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.341279 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.341242    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.341425 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.341392    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b429c098-3403-4f95-9052-fe7a8b995170-node-exporter-tls\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.349317 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.349273    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp9jd\" (UniqueName: \"kubernetes.io/projected/b429c098-3403-4f95-9052-fe7a8b995170-kube-api-access-fp9jd\") pod \"node-exporter-spmxg\" (UID: \"b429c098-3403-4f95-9052-fe7a8b995170\") " pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.453399 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:28.453359    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-spmxg"
Jul 18 01:55:28.822688 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:28.822656    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb429c098_3403_4f95_9052_fe7a8b995170.slice/crio-452eb0eff14feb79a8c7b5ec66e6ccfb12dde1826a712c23d91b56ef597f5647 WatchSource:0}: Error finding container 452eb0eff14feb79a8c7b5ec66e6ccfb12dde1826a712c23d91b56ef597f5647: Status 404 returned error can't find the container with id 452eb0eff14feb79a8c7b5ec66e6ccfb12dde1826a712c23d91b56ef597f5647
Jul 18 01:55:29.076595 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.076492    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b669c55b-xvrhm" event={"ID":"a00c655a-c222-4203-b371-5da034253e34","Type":"ContainerStarted","Data":"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"}
Jul 18 01:55:29.077744 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.077722    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spmxg" event={"ID":"b429c098-3403-4f95-9052-fe7a8b995170","Type":"ContainerStarted","Data":"452eb0eff14feb79a8c7b5ec66e6ccfb12dde1826a712c23d91b56ef597f5647"}
Jul 18 01:55:29.094786 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.094734    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b669c55b-xvrhm" podStartSLOduration=1.43720053 podStartE2EDuration="4.09471574s" totalImagesPullingTime="2.65751521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:29.093858783 +0000 UTC m=+298.416238949" watchObservedRunningTime="2026-07-18 01:55:29.09471574 +0000 UTC m=+298.417095908"
Jul 18 01:55:29.138912 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.138880    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:55:29.144694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.144657    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.147225 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147194    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 01:55:29.147225 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147198    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147264    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147270    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zvlfn\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147204    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147204    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147338    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 01:55:29.147435 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147270    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 01:55:29.147706 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147693    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 01:55:29.147748 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.147708    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 01:55:29.159036 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.158999    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:55:29.247128 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247081    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247175    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247220    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247251    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247278    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247466 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247350    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bglz\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247466 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247409    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247466 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247430    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247466 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247633 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247633 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247491    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.247633 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.247513    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348669 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348570    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348669 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348676    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bglz\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348764    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348788    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:29.348800    2560 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348814    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348842    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.348877 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.349310 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:29.348891    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls podName:181aadd6-ba89-4852-88ad-95450c8c284f nodeName:}" failed. No retries permitted until 2026-07-18 01:55:29.848867209 +0000 UTC m=+299.171247372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f") : secret "alertmanager-main-tls" not found
Jul 18 01:55:29.349310 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.348960    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.349310 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.349038    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.349310 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.349119    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.349610 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.349579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.350046 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.349784    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.350422 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.350394    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.351415 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.351387    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.352932 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.352891    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.353040 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.352969    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.353040 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.352972    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.353326 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.353308    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.353415 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.353347    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.353415 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.353358    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.354366 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.354347    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.356841 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.356823    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bglz\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.852463 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.852419    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:29.857091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:29.857031    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:30.054432 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:30.054397    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:55:30.055959 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:30.055933    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tltqd"
Jul 18 01:55:30.086378 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:30.086118    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spmxg" event={"ID":"b429c098-3403-4f95-9052-fe7a8b995170","Type":"ContainerStarted","Data":"fdb778ebd72251a82d565c7f6d8819f0f0bbafec45cb9d95298a85a525e25452"}
Jul 18 01:55:30.211642 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:30.211623    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:55:30.214109 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:30.214077    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod181aadd6_ba89_4852_88ad_95450c8c284f.slice/crio-dc074aead4eea0d2387254fa4aead6616dd9a453dc1aedeb6c90017c23d5f60e WatchSource:0}: Error finding container dc074aead4eea0d2387254fa4aead6616dd9a453dc1aedeb6c90017c23d5f60e: Status 404 returned error can't find the container with id dc074aead4eea0d2387254fa4aead6616dd9a453dc1aedeb6c90017c23d5f60e
Jul 18 01:55:31.092657 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:31.092436    2560 generic.go:356] "Generic (PLEG): container finished" podID="b429c098-3403-4f95-9052-fe7a8b995170" containerID="fdb778ebd72251a82d565c7f6d8819f0f0bbafec45cb9d95298a85a525e25452" exitCode=0
Jul 18 01:55:31.092657 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:31.092551    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spmxg" event={"ID":"b429c098-3403-4f95-9052-fe7a8b995170","Type":"ContainerDied","Data":"fdb778ebd72251a82d565c7f6d8819f0f0bbafec45cb9d95298a85a525e25452"}
Jul 18 01:55:31.094211 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:31.094189    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"dc074aead4eea0d2387254fa4aead6616dd9a453dc1aedeb6c90017c23d5f60e"}
Jul 18 01:55:31.255408 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:31.255375    2560 kubelet.go:1648] "Image garbage collection succeeded"
Jul 18 01:55:32.099837 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.099803    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spmxg" event={"ID":"b429c098-3403-4f95-9052-fe7a8b995170","Type":"ContainerStarted","Data":"363266793ec420f8a58465eb1c2c2a6b719d5153fda9ed52232a635af63e80d1"}
Jul 18 01:55:32.100039 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.099856    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-spmxg" event={"ID":"b429c098-3403-4f95-9052-fe7a8b995170","Type":"ContainerStarted","Data":"e0a8954847f000a90dfa86d3388d1250c9219fa40c0452c27a12833183cf2437"}
Jul 18 01:55:32.101381 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.101356    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="67b78639fda0a9c03b8aef7db0fcd9e95100e6bed2b8b01b51cb371a5466b0a4" exitCode=0
Jul 18 01:55:32.101480 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.101396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"67b78639fda0a9c03b8aef7db0fcd9e95100e6bed2b8b01b51cb371a5466b0a4"}
Jul 18 01:55:32.124943 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.124887    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-spmxg" podStartSLOduration=2.908992761 podStartE2EDuration="4.124870721s" totalImagesPullingTime="1.136017771s" totalInitContainerRuntime="79.860189ms" isStatefulPod=true podCreationTimestamp="2026-07-18 01:55:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:32.123314706 +0000 UTC m=+301.445694917" watchObservedRunningTime="2026-07-18 01:55:32.124870721 +0000 UTC m=+301.447250887"
Jul 18 01:55:32.399350 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.399312    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"]
Jul 18 01:55:32.403260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.403236    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.405874 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.405817    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-478g2bcob6qbs\""
Jul 18 01:55:32.406032 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.405918    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 18 01:55:32.406032 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.405917    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 18 01:55:32.406032 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.405937    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 18 01:55:32.406266 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.406171    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-t27jr\""
Jul 18 01:55:32.406565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.406550    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 18 01:55:32.412144 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.412121    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"]
Jul 18 01:55:32.478650 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478618    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-tls\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.478848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.478848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478751    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-client-certs\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.478848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478784    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-client-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.478848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16550513-a742-4c93-84b8-dae085242159-audit-log\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.478848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478829    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-metrics-server-audit-profiles\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.479025 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.478896    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72m6v\" (UniqueName: \"kubernetes.io/projected/16550513-a742-4c93-84b8-dae085242159-kube-api-access-72m6v\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580043    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580285 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580184    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-client-certs\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580285 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580232    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-client-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580285 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580264    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16550513-a742-4c93-84b8-dae085242159-audit-log\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580441 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580295    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-metrics-server-audit-profiles\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580441 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580339    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72m6v\" (UniqueName: \"kubernetes.io/projected/16550513-a742-4c93-84b8-dae085242159-kube-api-access-72m6v\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580441 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580405    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-tls\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580758 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580726    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/16550513-a742-4c93-84b8-dae085242159-audit-log\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.580891 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.580833    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.581297 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.581280    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/16550513-a742-4c93-84b8-dae085242159-metrics-server-audit-profiles\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.582959 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.582940    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-client-certs\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.583118 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.583098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-client-ca-bundle\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.583154 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.583098    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/16550513-a742-4c93-84b8-dae085242159-secret-metrics-server-tls\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.590659 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.590631    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72m6v\" (UniqueName: \"kubernetes.io/projected/16550513-a742-4c93-84b8-dae085242159-kube-api-access-72m6v\") pod \"metrics-server-5bf7cb7d99-gc2cz\" (UID: \"16550513-a742-4c93-84b8-dae085242159\") " pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.712818 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.712730    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:32.865267 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.865233    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"]
Jul 18 01:55:32.869677 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.869644    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"]
Jul 18 01:55:32.869776 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.869762    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:32.872220 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.872186    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-km6sc\""
Jul 18 01:55:32.872327 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.872231    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 18 01:55:32.872795 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:32.872773    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16550513_a742_4c93_84b8_dae085242159.slice/crio-c61a6905ac8fbd73669238c71bc6d311772f7fe871935e465b7c89d486fdc2bd WatchSource:0}: Error finding container c61a6905ac8fbd73669238c71bc6d311772f7fe871935e465b7c89d486fdc2bd: Status 404 returned error can't find the container with id c61a6905ac8fbd73669238c71bc6d311772f7fe871935e465b7c89d486fdc2bd
Jul 18 01:55:32.876145 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.875642    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 01:55:32.876215 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.876192    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"]
Jul 18 01:55:32.984824 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.984730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:32.984988 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:32.984868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dpkw\" (UniqueName: \"kubernetes.io/projected/42a460ee-1480-45e6-b0b5-046853c97a7a-kube-api-access-7dpkw\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.086264 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.086229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dpkw\" (UniqueName: \"kubernetes.io/projected/42a460ee-1480-45e6-b0b5-046853c97a7a-kube-api-access-7dpkw\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.086423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.086272    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.086423 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:33.086397    2560 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 18 01:55:33.086488 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:55:33.086447    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert podName:42a460ee-1480-45e6-b0b5-046853c97a7a nodeName:}" failed. No retries permitted until 2026-07-18 01:55:33.586433139 +0000 UTC m=+302.908813283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-99895" (UID: "42a460ee-1480-45e6-b0b5-046853c97a7a") : secret "monitoring-plugin-cert" not found
Jul 18 01:55:33.094413 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.094378    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dpkw\" (UniqueName: \"kubernetes.io/projected/42a460ee-1480-45e6-b0b5-046853c97a7a-kube-api-access-7dpkw\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.116929 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.116897    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz" event={"ID":"16550513-a742-4c93-84b8-dae085242159","Type":"ContainerStarted","Data":"c61a6905ac8fbd73669238c71bc6d311772f7fe871935e465b7c89d486fdc2bd"}
Jul 18 01:55:33.284133 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.284045    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"]
Jul 18 01:55:33.289091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.289063    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.291694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.291508    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 18 01:55:33.291694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.291578    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 18 01:55:33.291694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.291597    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bnppf\""
Jul 18 01:55:33.291694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.291601    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 18 01:55:33.292270 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.292246    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 18 01:55:33.292627 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.292607    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 18 01:55:33.297126 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.297088    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 18 01:55:33.302573 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.302545    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"]
Jul 18 01:55:33.388746 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388714    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-serving-certs-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.388746 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388752    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-federate-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.388974 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388790    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.388974 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388876    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.388974 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388942    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n4j4\" (UniqueName: \"kubernetes.io/projected/0cfe2ce3-d22a-4f46-919d-955e58c93673-kube-api-access-7n4j4\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.389117 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388977    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.389117 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.388995    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-metrics-client-ca\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.389117 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.389034    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.489694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489667    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489708    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-metrics-client-ca\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489737    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489783    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-serving-certs-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489799    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-federate-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489823    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489859    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490030 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.489912    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n4j4\" (UniqueName: \"kubernetes.io/projected/0cfe2ce3-d22a-4f46-919d-955e58c93673-kube-api-access-7n4j4\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490769 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.490744    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-metrics-client-ca\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.490890 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.490795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-serving-certs-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.491212 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.491182    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.492432 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.492346    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.492563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.492540    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-telemeter-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.492692 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.492677    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-federate-client-tls\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.492968 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.492951    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0cfe2ce3-d22a-4f46-919d-955e58c93673-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.498405 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.498380    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n4j4\" (UniqueName: \"kubernetes.io/projected/0cfe2ce3-d22a-4f46-919d-955e58c93673-kube-api-access-7n4j4\") pod \"telemeter-client-76dd4c5b74-zv99p\" (UID: \"0cfe2ce3-d22a-4f46-919d-955e58c93673\") " pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.590968 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.590919    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.593838 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.593813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/42a460ee-1480-45e6-b0b5-046853c97a7a-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-99895\" (UID: \"42a460ee-1480-45e6-b0b5-046853c97a7a\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.602274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.602245    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"
Jul 18 01:55:33.760899 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.760870    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:55:33.765624 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.765599    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.772162 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.771971    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76dd4c5b74-zv99p"]
Jul 18 01:55:33.772991 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.772944    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 18 01:55:33.774621 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.774570    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:55:33.777589 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:33.777560    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cfe2ce3_d22a_4f46_919d_955e58c93673.slice/crio-0b469fe65535a79d058471816efd3b02f88e5e4d16468272d950f0fc58650fee WatchSource:0}: Error finding container 0b469fe65535a79d058471816efd3b02f88e5e4d16468272d950f0fc58650fee: Status 404 returned error can't find the container with id 0b469fe65535a79d058471816efd3b02f88e5e4d16468272d950f0fc58650fee
Jul 18 01:55:33.783713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.783688    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:33.792765 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.792863 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792773    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggct\" (UniqueName: \"kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.792863 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792800    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.792863 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792860    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.793026 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792928    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.793026 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.792968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.793026 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.793012    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894144 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894104    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894303 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894163    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894303 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894210    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894303 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894268    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894456 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894304    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggct\" (UniqueName: \"kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894456 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894333    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.894456 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.894365    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.895224 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.895167    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.895966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.895377    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.895966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.895763    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.895966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.895930    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.898987 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.898906    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.898987 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.898970    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.903416 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.903365    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggct\" (UniqueName: \"kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct\") pod \"console-77656fb6d-jsbb6\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") " pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:33.948156 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:33.948124    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"]
Jul 18 01:55:33.951236 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:33.951204    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a460ee_1480_45e6_b0b5_046853c97a7a.slice/crio-8dade2ce6a79207ff7010b330adf20e7ab0217ad8264b3e892f58fdcd87c037c WatchSource:0}: Error finding container 8dade2ce6a79207ff7010b330adf20e7ab0217ad8264b3e892f58fdcd87c037c: Status 404 returned error can't find the container with id 8dade2ce6a79207ff7010b330adf20e7ab0217ad8264b3e892f58fdcd87c037c
Jul 18 01:55:34.096742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.096646    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:34.119217 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.119169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895" event={"ID":"42a460ee-1480-45e6-b0b5-046853c97a7a","Type":"ContainerStarted","Data":"8dade2ce6a79207ff7010b330adf20e7ab0217ad8264b3e892f58fdcd87c037c"}
Jul 18 01:55:34.123418 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.123249    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"cbe9f3509f4e2b7c79443703df09a36abb8331a11cf9eca17014733991b11b63"}
Jul 18 01:55:34.123418 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.123326    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"cd4ababa50540f11186bf2fa6dafda97be19601a856d64e2f5008128de60b0c4"}
Jul 18 01:55:34.123418 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.123362    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"061ea66e3afa36bd0ae3d1425b00fd243ee16f0d1cd19d94c48aa5e659848b2a"}
Jul 18 01:55:34.123418 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.123401    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"cbe946e029b87c45e1b7a65d651a85d6fc226c407656616ee0d039498ab9e58c"}
Jul 18 01:55:34.123799 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.123436    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"af4931f018297bc1cdaa43f268b779404591fe983c40deaa2123fd6781fdaab6"}
Jul 18 01:55:34.124882 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.124807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p" event={"ID":"0cfe2ce3-d22a-4f46-919d-955e58c93673","Type":"ContainerStarted","Data":"0b469fe65535a79d058471816efd3b02f88e5e4d16468272d950f0fc58650fee"}
Jul 18 01:55:34.263085 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.262301    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:55:34.273319 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.273287    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:55:34.287401 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.287363    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.290302 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.290276    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 01:55:34.290568 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.290356    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 01:55:34.290703 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.290682    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-293gs02thg94l\""
Jul 18 01:55:34.291425 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.291386    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 01:55:34.291704 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.291686    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.291957    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.294918    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295177    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295372    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295450    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295620    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295668    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295830    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.295850    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zzr28\""
Jul 18 01:55:34.297143 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.296115    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 01:55:34.298731 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.298606    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:55:34.300232 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.300213    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 01:55:34.399331 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399296    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399509 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399346    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399509 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399509 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399394    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399509 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399473    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399723 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399723 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399640    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399723 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399670    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94n7l\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399723 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399867 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399867 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399757    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399867 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399858    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399994 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399898    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399994 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.399994 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.399978    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.400092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.400017    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.400092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.400060    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.400185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.400100    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.400185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.400124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.400185 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.400158    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500681    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500726    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500760    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500802    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500834    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500855    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94n7l\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500914    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.500937    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501030    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501051    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501075    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501101    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501142    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501180    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.501966 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501204    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501229    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501252    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.501279    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.502559    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.502757    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503254 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.502862    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.503817 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.503575    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.504854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.504822    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.506170 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.506145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.506372 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.506349    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.507082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.507051    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.507293 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.507266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.507939 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.507704    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.507939 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.507792    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.508082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.507949    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.508450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.508235    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.508450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.508385    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.508720 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.508648    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.508720 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.508652    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.509713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.509693    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.509925 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.509901    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.510445 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.510428    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.511431 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.511410    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94n7l\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l\") pod \"prometheus-k8s-0\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.605326 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.605273    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:34.608807 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:34.608359    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac7b291_6fd7_40b0_95f0_f5430a238ba0.slice/crio-a1119a96d85d719e930dc90a9d674233094ae1ae3535cf171f776dcbdf33fa90 WatchSource:0}: Error finding container a1119a96d85d719e930dc90a9d674233094ae1ae3535cf171f776dcbdf33fa90: Status 404 returned error can't find the container with id a1119a96d85d719e930dc90a9d674233094ae1ae3535cf171f776dcbdf33fa90
Jul 18 01:55:34.831189 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:34.830429    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:55:34.847960 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:34.846155    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53b02bb1_48bd_4d06_8364_19f4902ef341.slice/crio-55e2860d12fdfc4183d35769a39530fee6bf9b4d29e75ddcbf27ec716ab03b4b WatchSource:0}: Error finding container 55e2860d12fdfc4183d35769a39530fee6bf9b4d29e75ddcbf27ec716ab03b4b: Status 404 returned error can't find the container with id 55e2860d12fdfc4183d35769a39530fee6bf9b4d29e75ddcbf27ec716ab03b4b
Jul 18 01:55:35.132616 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.132563    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"}
Jul 18 01:55:35.132786 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.132655    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"55e2860d12fdfc4183d35769a39530fee6bf9b4d29e75ddcbf27ec716ab03b4b"}
Jul 18 01:55:35.136712 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.136149    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz" event={"ID":"16550513-a742-4c93-84b8-dae085242159","Type":"ContainerStarted","Data":"ad81c79bba2ab61bb171314f37c67d07d28d9a279f5511d229bc928209c72719"}
Jul 18 01:55:35.140178 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.140095    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77656fb6d-jsbb6" event={"ID":"2ac7b291-6fd7-40b0-95f0-f5430a238ba0","Type":"ContainerStarted","Data":"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"}
Jul 18 01:55:35.140178 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.140147    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77656fb6d-jsbb6" event={"ID":"2ac7b291-6fd7-40b0-95f0-f5430a238ba0","Type":"ContainerStarted","Data":"a1119a96d85d719e930dc90a9d674233094ae1ae3535cf171f776dcbdf33fa90"}
Jul 18 01:55:35.174405 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.174218    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz" podStartSLOduration=1.3743859569999999 podStartE2EDuration="3.174193907s" totalImagesPullingTime="1.79980795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:35.173139026 +0000 UTC m=+304.495519194" watchObservedRunningTime="2026-07-18 01:55:35.174193907 +0000 UTC m=+304.496574073"
Jul 18 01:55:35.189424 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:35.189363    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77656fb6d-jsbb6" podStartSLOduration=2.189342434 podStartE2EDuration="2.189342434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:35.188605433 +0000 UTC m=+304.510985599" watchObservedRunningTime="2026-07-18 01:55:35.189342434 +0000 UTC m=+304.511722601"
Jul 18 01:55:36.074919 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.074849    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:36.074919 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.074927    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:36.080462 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.080432    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:36.148209 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.148168    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerStarted","Data":"685906cd9427f71692f405a2a7b2e26c1bd44490ef2d2634142e53b44cd5a262"}
Jul 18 01:55:36.149922 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.149896    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18" exitCode=0
Jul 18 01:55:36.150044 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.149991    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"}
Jul 18 01:55:36.154719 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.154694    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:55:36.175955 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:36.175894    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.438028922 podStartE2EDuration="7.175867893s" totalImagesPullingTime="3.721878407s" totalInitContainerRuntime="15.960564ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:36.173886733 +0000 UTC m=+305.496266914" watchObservedRunningTime="2026-07-18 01:55:36.175867893 +0000 UTC m=+305.498248064"
Jul 18 01:55:37.157171 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.157136    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895" event={"ID":"42a460ee-1480-45e6-b0b5-046853c97a7a","Type":"ContainerStarted","Data":"ccc5006a197d1c8b650a57c51adc5491b3dc21c32beeaf68c9cd4df2e06d9bdc"}
Jul 18 01:55:37.157690 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.157325    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:37.160678 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.160439    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p" event={"ID":"0cfe2ce3-d22a-4f46-919d-955e58c93673","Type":"ContainerStarted","Data":"2217e318c725d04436df29d483cdb738d15e359d84ae4e942c50eb94c46c93cc"}
Jul 18 01:55:37.160678 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.160511    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p" event={"ID":"0cfe2ce3-d22a-4f46-919d-955e58c93673","Type":"ContainerStarted","Data":"43cf710d2e1caa490e410ee6286b59df325fe1dbaa0edfaaf938bdb4951ddbf1"}
Jul 18 01:55:37.160678 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.160568    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p" event={"ID":"0cfe2ce3-d22a-4f46-919d-955e58c93673","Type":"ContainerStarted","Data":"38da2af22d48979ba217aad372e2f9d930fd44a981aab9eaf4865b48f1946f97"}
Jul 18 01:55:37.164159 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.164137    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895"
Jul 18 01:55:37.173862 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.173792    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-99895" podStartSLOduration=2.863733091 podStartE2EDuration="5.173769263s" totalImagesPullingTime="2.310036172s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:37.1722888 +0000 UTC m=+306.494668969" watchObservedRunningTime="2026-07-18 01:55:37.173769263 +0000 UTC m=+306.496149430"
Jul 18 01:55:37.212001 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.211942    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76dd4c5b74-zv99p" podStartSLOduration=1.724253349 podStartE2EDuration="4.211918495s" totalImagesPullingTime="2.487665146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:37.210231913 +0000 UTC m=+306.532612080" watchObservedRunningTime="2026-07-18 01:55:37.211918495 +0000 UTC m=+306.534298660"
Jul 18 01:55:37.312540 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.312474    2560 patch_prober.go:28] interesting pod/image-registry-69f6bf6697-jg9ch container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 18 01:55:37.314794 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:37.312570    2560 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 18 01:55:39.689581 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.689541    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:55:39.715277 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.715245    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:55:39.718717 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.718701    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.727729 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.727705    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:55:39.863368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863329    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863605 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863605 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkbtq\" (UniqueName: \"kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863605 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863581    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863761 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.863761 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.863691    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965181 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965091    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965181 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965135    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965216    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkbtq\" (UniqueName: \"kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965244    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965291    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965345    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.965403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965373    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.966019 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965990    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.966019 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.965990    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.966321 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.966255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.966452 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.966425    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.968247 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.968226    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.968468 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.968445    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:39.973097 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:39.973073    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkbtq\" (UniqueName: \"kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq\") pod \"console-67796f69fd-dhpjh\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") " pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:40.028290 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:40.028246    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:40.183130 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:40.183003    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"}
Jul 18 01:55:40.183130 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:40.183082    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"}
Jul 18 01:55:40.185775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:40.185746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:55:40.188234 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:55:40.188206    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode535e150_d0e8_4f56_a83f_aee5fba9df5b.slice/crio-22d8788e13f62dd45b05830eeb1909668a366f3370751a4ae7c8616e802796ce WatchSource:0}: Error finding container 22d8788e13f62dd45b05830eeb1909668a366f3370751a4ae7c8616e802796ce: Status 404 returned error can't find the container with id 22d8788e13f62dd45b05830eeb1909668a366f3370751a4ae7c8616e802796ce
Jul 18 01:55:41.043688 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.043662    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:55:41.113230 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.113204    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:55:41.189894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.189861    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67796f69fd-dhpjh" event={"ID":"e535e150-d0e8-4f56-a83f-aee5fba9df5b","Type":"ContainerStarted","Data":"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"}
Jul 18 01:55:41.189984 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.189927    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67796f69fd-dhpjh" event={"ID":"e535e150-d0e8-4f56-a83f-aee5fba9df5b","Type":"ContainerStarted","Data":"22d8788e13f62dd45b05830eeb1909668a366f3370751a4ae7c8616e802796ce"}
Jul 18 01:55:41.213742 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.213682    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67796f69fd-dhpjh" podStartSLOduration=2.213665898 podStartE2EDuration="2.213665898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:41.212117439 +0000 UTC m=+310.534497625" watchObservedRunningTime="2026-07-18 01:55:41.213665898 +0000 UTC m=+310.536046063"
Jul 18 01:55:41.752811 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:41.752779    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:55:42.197684 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:42.197650    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"}
Jul 18 01:55:42.198049 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:42.197719    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"}
Jul 18 01:55:42.198049 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:42.197748    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"}
Jul 18 01:55:42.198049 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:42.197773    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerStarted","Data":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"}
Jul 18 01:55:42.228257 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:42.228198    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.351419695 podStartE2EDuration="8.228176142s" totalImagesPullingTime="4.853183464s" totalInitContainerRuntime="23.572983ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:55:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:55:42.226722541 +0000 UTC m=+311.549102736" watchObservedRunningTime="2026-07-18 01:55:42.228176142 +0000 UTC m=+311.550556309"
Jul 18 01:55:44.096831 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:44.096787    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:55:44.606340 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:44.606303    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:55:50.028948 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:50.028913    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:50.029472 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:50.028978    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:50.035084 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:50.035063    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:50.239220 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:50.239186    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:55:50.282367 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:50.282285    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:55:52.712829 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:52.712791    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:55:52.713313 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:55:52.712857    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:56:04.710217 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:04.710172    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77656fb6d-jsbb6" podUID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" containerName="console" containerID="cri-o://6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea" gracePeriod=15
Jul 18 01:56:04.952286 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:04.952262    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:56:05.005258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005163    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005258 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005242    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005474 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005278    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005474 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005312    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005474 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005357    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fggct\" (UniqueName: \"kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005474 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005380    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005474 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005405    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert\") pod \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\" (UID: \"2ac7b291-6fd7-40b0-95f0-f5430a238ba0\") "
Jul 18 01:56:05.005775 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005747    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:05.005843 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005796    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:05.005901 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005871    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config" (OuterVolumeSpecName: "console-config") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:05.005951 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.005910    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca" (OuterVolumeSpecName: "service-ca") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:05.007826 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.007798    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:05.007961 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.007880    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct" (OuterVolumeSpecName: "kube-api-access-fggct") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "kube-api-access-fggct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:05.007961 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.007888    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2ac7b291-6fd7-40b0-95f0-f5430a238ba0" (UID: "2ac7b291-6fd7-40b0-95f0-f5430a238ba0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:05.107089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107049    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-oauth-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107080    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-oauth-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107089    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107089 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107097    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fggct\" (UniqueName: \"kubernetes.io/projected/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-kube-api-access-fggct\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107353 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107105    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107353 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107114    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-console-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.107353 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.107122    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2ac7b291-6fd7-40b0-95f0-f5430a238ba0-trusted-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:05.301843 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.301764    2560 generic.go:356] "Generic (PLEG): container finished" podID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" containerID="6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea" exitCode=0
Jul 18 01:56:05.301843 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.301814    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77656fb6d-jsbb6" event={"ID":"2ac7b291-6fd7-40b0-95f0-f5430a238ba0","Type":"ContainerDied","Data":"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"}
Jul 18 01:56:05.301843 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.301826    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77656fb6d-jsbb6"
Jul 18 01:56:05.302081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.301854    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77656fb6d-jsbb6" event={"ID":"2ac7b291-6fd7-40b0-95f0-f5430a238ba0","Type":"ContainerDied","Data":"a1119a96d85d719e930dc90a9d674233094ae1ae3535cf171f776dcbdf33fa90"}
Jul 18 01:56:05.302081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.301875    2560 scope.go:117] "RemoveContainer" containerID="6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"
Jul 18 01:56:05.310653 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.310634    2560 scope.go:117] "RemoveContainer" containerID="6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"
Jul 18 01:56:05.310911 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:05.310894    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea\": container with ID starting with 6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea not found: ID does not exist" containerID="6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"
Jul 18 01:56:05.310958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.310919    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea"} err="failed to get container status \"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea\": rpc error: code = NotFound desc = could not find container \"6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea\": container with ID starting with 6c01a713606072be921a8e164271e09ba50bc1e51769d5de36d3d1ddb82ea8ea not found: ID does not exist"
Jul 18 01:56:05.323659 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.323632    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:56:05.331051 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:05.331022    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77656fb6d-jsbb6"]
Jul 18 01:56:06.773660 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:06.773596    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerName="registry" containerID="cri-o://ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2" gracePeriod=30
Jul 18 01:56:07.030861 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.030797    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:56:07.115751 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.115720    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" path="/var/lib/kubelet/pods/2ac7b291-6fd7-40b0-95f0-f5430a238ba0/volumes"
Jul 18 01:56:07.126752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126730    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.126871 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126763    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.126871 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126852    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.126978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126884    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxfw5\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.126978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126919    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.126978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.126961    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.127128 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127002    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.127128 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127029    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets\") pod \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\" (UID: \"a10a0f1c-7ae9-42aa-8422-751679b17d8f\") "
Jul 18 01:56:07.127225 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127160    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:07.127320 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127295    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:07.127464 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127448    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-trusted-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.127546 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.127470    2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-certificates\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.129764 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.129729    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:07.129926 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.129892    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5" (OuterVolumeSpecName: "kube-api-access-mxfw5") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "kube-api-access-mxfw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:07.129926 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.129907    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:07.130086 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.129970    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:07.130086 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.130044    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:07.135669 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.135634    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a10a0f1c-7ae9-42aa-8422-751679b17d8f" (UID: "a10a0f1c-7ae9-42aa-8422-751679b17d8f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:07.228467 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228428    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxfw5\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-kube-api-access-mxfw5\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.228467 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228458    2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a10a0f1c-7ae9-42aa-8422-751679b17d8f-ca-trust-extracted\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.228467 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228467    2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-bound-sa-token\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.228467 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228475    2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a10a0f1c-7ae9-42aa-8422-751679b17d8f-registry-tls\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.228744 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228483    2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-installation-pull-secrets\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.228744 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.228492    2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a10a0f1c-7ae9-42aa-8422-751679b17d8f-image-registry-private-configuration\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:07.312708 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.312625    2560 generic.go:356] "Generic (PLEG): container finished" podID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerID="ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2" exitCode=0
Jul 18 01:56:07.312708 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.312657    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch"
Jul 18 01:56:07.312708 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.312696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" event={"ID":"a10a0f1c-7ae9-42aa-8422-751679b17d8f","Type":"ContainerDied","Data":"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"}
Jul 18 01:56:07.312978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.312756    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-69f6bf6697-jg9ch" event={"ID":"a10a0f1c-7ae9-42aa-8422-751679b17d8f","Type":"ContainerDied","Data":"cd74f01f45006321244fecae6f7c1301eda332214feffb2922c342b03558de80"}
Jul 18 01:56:07.312978 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.312784    2560 scope.go:117] "RemoveContainer" containerID="ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"
Jul 18 01:56:07.322496 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.322479    2560 scope.go:117] "RemoveContainer" containerID="ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"
Jul 18 01:56:07.322836 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:07.322817    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2\": container with ID starting with ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2 not found: ID does not exist" containerID="ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"
Jul 18 01:56:07.322880 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.322846    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2"} err="failed to get container status \"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2\": rpc error: code = NotFound desc = could not find container \"ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2\": container with ID starting with ae269f173dff2774d38a34ef0ffb93f7b74a1e0ca0bda046060bd4ba2401cbb2 not found: ID does not exist"
Jul 18 01:56:07.334567 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.334537    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:56:07.338357 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:07.338334    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-69f6bf6697-jg9ch"]
Jul 18 01:56:09.116302 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:09.116266    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" path="/var/lib/kubelet/pods/a10a0f1c-7ae9-42aa-8422-751679b17d8f/volumes"
Jul 18 01:56:12.718347 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:12.718313    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:56:12.722224 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:12.722197    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5bf7cb7d99-gc2cz"
Jul 18 01:56:15.307802 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.307732    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b669c55b-xvrhm" podUID="a00c655a-c222-4203-b371-5da034253e34" containerName="console" containerID="cri-o://3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e" gracePeriod=15
Jul 18 01:56:15.557504 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.557477    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:56:15.606216 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606116    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606216 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606170    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606221    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606250    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606281    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddc7x\" (UniqueName: \"kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606319    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config\") pod \"a00c655a-c222-4203-b371-5da034253e34\" (UID: \"a00c655a-c222-4203-b371-5da034253e34\") "
Jul 18 01:56:15.606694 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606664    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:15.606762 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606672    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca" (OuterVolumeSpecName: "service-ca") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:15.606816 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.606776    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config" (OuterVolumeSpecName: "console-config") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:15.608637 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.608481    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:15.608637 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.608534    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x" (OuterVolumeSpecName: "kube-api-access-ddc7x") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "kube-api-access-ddc7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:15.608637 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.608562    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a00c655a-c222-4203-b371-5da034253e34" (UID: "a00c655a-c222-4203-b371-5da034253e34"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:15.707958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707920    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-console-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:15.707958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707947    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:15.707958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707957    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-oauth-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:15.707958 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707966    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a00c655a-c222-4203-b371-5da034253e34-console-oauth-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:15.708239 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707976    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a00c655a-c222-4203-b371-5da034253e34-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:15.708239 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:15.707984    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddc7x\" (UniqueName: \"kubernetes.io/projected/a00c655a-c222-4203-b371-5da034253e34-kube-api-access-ddc7x\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:16.355633 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.355598    2560 generic.go:356] "Generic (PLEG): container finished" podID="a00c655a-c222-4203-b371-5da034253e34" containerID="3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e" exitCode=0
Jul 18 01:56:16.356060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.355647    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b669c55b-xvrhm"
Jul 18 01:56:16.356060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.355688    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b669c55b-xvrhm" event={"ID":"a00c655a-c222-4203-b371-5da034253e34","Type":"ContainerDied","Data":"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"}
Jul 18 01:56:16.356060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.355741    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b669c55b-xvrhm" event={"ID":"a00c655a-c222-4203-b371-5da034253e34","Type":"ContainerDied","Data":"dfbd8c6d3b5c110b9f62318cad72f6cfeb0e97c3b8404655087e2dc5d32ac014"}
Jul 18 01:56:16.356060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.355762    2560 scope.go:117] "RemoveContainer" containerID="3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"
Jul 18 01:56:16.364814 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.364796    2560 scope.go:117] "RemoveContainer" containerID="3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"
Jul 18 01:56:16.365101 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:16.365067    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e\": container with ID starting with 3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e not found: ID does not exist" containerID="3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"
Jul 18 01:56:16.365161 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.365114    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e"} err="failed to get container status \"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e\": rpc error: code = NotFound desc = could not find container \"3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e\": container with ID starting with 3e7ddb38729ec0a78a8cd5a2dec0e91acf45c2e3df44b0908994082be61d315e not found: ID does not exist"
Jul 18 01:56:16.375835 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.375809    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:56:16.378008 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:16.377984    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b669c55b-xvrhm"]
Jul 18 01:56:17.115555 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:17.115501    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a00c655a-c222-4203-b371-5da034253e34" path="/var/lib/kubelet/pods/a00c655a-c222-4203-b371-5da034253e34/volumes"
Jul 18 01:56:22.383984 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:22.383947    2560 generic.go:356] "Generic (PLEG): container finished" podID="d8f740af-542b-40c4-be19-372fe4ed6873" containerID="e0d2935a9f7c1cdd772688bf7927641875ad7d28b61a7bd47fe21dfb0e888448" exitCode=0
Jul 18 01:56:22.384460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:22.384009    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44" event={"ID":"d8f740af-542b-40c4-be19-372fe4ed6873","Type":"ContainerDied","Data":"e0d2935a9f7c1cdd772688bf7927641875ad7d28b61a7bd47fe21dfb0e888448"}
Jul 18 01:56:22.384460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:22.384416    2560 scope.go:117] "RemoveContainer" containerID="e0d2935a9f7c1cdd772688bf7927641875ad7d28b61a7bd47fe21dfb0e888448"
Jul 18 01:56:23.389989 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:23.389957    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-t4m44" event={"ID":"d8f740af-542b-40c4-be19-372fe4ed6873","Type":"ContainerStarted","Data":"3f6395c43beb41e890be56b9297a5cbebd8caa9a1a9c04fe3872f226856011a4"}
Jul 18 01:56:34.606228 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:34.606198    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:34.622620 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:34.622594    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:35.474045 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:35.474020    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:44.977000 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:44.976967    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:56:44.979417 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:44.979389    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e-metrics-certs\") pod \"network-metrics-daemon-6r66r\" (UID: \"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e\") " pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:56:45.016281 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:45.016256    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rbznb\""
Jul 18 01:56:45.024150 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:45.024131    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6r66r"
Jul 18 01:56:45.161359 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:45.161311    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6r66r"]
Jul 18 01:56:45.165854 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:56:45.165825    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc284b922_9ec6_4f4e_8a2d_b3f99ace8b2e.slice/crio-6b2bfabbf59de850acdb09ab6be4fb72ecb8631431b807728a3e778dfaf7c51d WatchSource:0}: Error finding container 6b2bfabbf59de850acdb09ab6be4fb72ecb8631431b807728a3e778dfaf7c51d: Status 404 returned error can't find the container with id 6b2bfabbf59de850acdb09ab6be4fb72ecb8631431b807728a3e778dfaf7c51d
Jul 18 01:56:45.497883 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:45.497841    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r66r" event={"ID":"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e","Type":"ContainerStarted","Data":"6b2bfabbf59de850acdb09ab6be4fb72ecb8631431b807728a3e778dfaf7c51d"}
Jul 18 01:56:46.504470 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:46.504377    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r66r" event={"ID":"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e","Type":"ContainerStarted","Data":"dbd066d2f508c63dc7b6cbd802d14d4280f7f7b422ff4b0a029a6383b3d8c59c"}
Jul 18 01:56:46.504470 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:46.504463    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6r66r" event={"ID":"c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e","Type":"ContainerStarted","Data":"8c627e4fc65bc75e638b077be1015e384e51259951bb92f8ae2f8848da209c6a"}
Jul 18 01:56:46.520238 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:46.520181    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-6r66r" podStartSLOduration=373.544308026 podStartE2EDuration="6m14.520163959s" totalImagesPullingTime="975.855933ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:46.519661953 +0000 UTC m=+375.842042118" watchObservedRunningTime="2026-07-18 01:56:46.520163959 +0000 UTC m=+375.842544124"
Jul 18 01:56:48.388092 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388053    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:48.388565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388364    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="alertmanager" containerID="cri-o://af4931f018297bc1cdaa43f268b779404591fe983c40deaa2123fd6781fdaab6" gracePeriod=120
Jul 18 01:56:48.388565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388400    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="prom-label-proxy" containerID="cri-o://685906cd9427f71692f405a2a7b2e26c1bd44490ef2d2634142e53b44cd5a262" gracePeriod=120
Jul 18 01:56:48.388565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388431    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="config-reloader" containerID="cri-o://cbe946e029b87c45e1b7a65d651a85d6fc226c407656616ee0d039498ab9e58c" gracePeriod=120
Jul 18 01:56:48.388565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388420    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-web" containerID="cri-o://061ea66e3afa36bd0ae3d1425b00fd243ee16f0d1cd19d94c48aa5e659848b2a" gracePeriod=120
Jul 18 01:56:48.388565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388507    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy" containerID="cri-o://cd4ababa50540f11186bf2fa6dafda97be19601a856d64e2f5008128de60b0c4" gracePeriod=120
Jul 18 01:56:48.388830 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.388574    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-metric" containerID="cri-o://cbe9f3509f4e2b7c79443703df09a36abb8331a11cf9eca17014733991b11b63" gracePeriod=120
Jul 18 01:56:48.516659 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516630    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="685906cd9427f71692f405a2a7b2e26c1bd44490ef2d2634142e53b44cd5a262" exitCode=0
Jul 18 01:56:48.516659 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516656    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="cd4ababa50540f11186bf2fa6dafda97be19601a856d64e2f5008128de60b0c4" exitCode=0
Jul 18 01:56:48.516659 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516665    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="cbe946e029b87c45e1b7a65d651a85d6fc226c407656616ee0d039498ab9e58c" exitCode=0
Jul 18 01:56:48.516860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516671    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="af4931f018297bc1cdaa43f268b779404591fe983c40deaa2123fd6781fdaab6" exitCode=0
Jul 18 01:56:48.516860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516698    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"685906cd9427f71692f405a2a7b2e26c1bd44490ef2d2634142e53b44cd5a262"}
Jul 18 01:56:48.516860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516764    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"cd4ababa50540f11186bf2fa6dafda97be19601a856d64e2f5008128de60b0c4"}
Jul 18 01:56:48.516860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"cbe946e029b87c45e1b7a65d651a85d6fc226c407656616ee0d039498ab9e58c"}
Jul 18 01:56:48.516860 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:48.516821    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"af4931f018297bc1cdaa43f268b779404591fe983c40deaa2123fd6781fdaab6"}
Jul 18 01:56:49.526205 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.526172    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="cbe9f3509f4e2b7c79443703df09a36abb8331a11cf9eca17014733991b11b63" exitCode=0
Jul 18 01:56:49.526205 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.526202    2560 generic.go:356] "Generic (PLEG): container finished" podID="181aadd6-ba89-4852-88ad-95450c8c284f" containerID="061ea66e3afa36bd0ae3d1425b00fd243ee16f0d1cd19d94c48aa5e659848b2a" exitCode=0
Jul 18 01:56:49.526575 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.526235    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"cbe9f3509f4e2b7c79443703df09a36abb8331a11cf9eca17014733991b11b63"}
Jul 18 01:56:49.526575 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.526297    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"061ea66e3afa36bd0ae3d1425b00fd243ee16f0d1cd19d94c48aa5e659848b2a"}
Jul 18 01:56:49.646855 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.646827    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:49.717892 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.717811    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718018 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.717900    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718018 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.717970    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4bglz\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718018 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718003    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718173 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718035    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718173 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718092    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718173 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718123    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718173 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718156    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718178    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718191    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718233    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718237    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718289    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718316    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets\") pod \"181aadd6-ba89-4852-88ad-95450c8c284f\" (UID: \"181aadd6-ba89-4852-88ad-95450c8c284f\") "
Jul 18 01:56:49.718754 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718659    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.718754 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.718681    2560 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-alertmanager-main-db\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.722326 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.722256    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:49.722638 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.722367    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.723077 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.723036    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.723174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.723139    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.723328 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.723299    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz" (OuterVolumeSpecName: "kube-api-access-4bglz") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "kube-api-access-4bglz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:49.723764 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.723736    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume" (OuterVolumeSpecName: "config-volume") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.723845 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.723754    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:49.725356 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.725321    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.726422 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.726393    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out" (OuterVolumeSpecName: "config-out") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:49.728101 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.727876    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.736887 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.736853    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config" (OuterVolumeSpecName: "web-config") pod "181aadd6-ba89-4852-88ad-95450c8c284f" (UID: "181aadd6-ba89-4852-88ad-95450c8c284f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:49.819570 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819500    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bglz\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-kube-api-access-4bglz\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819570 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819564    2560 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-config-volume\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819570 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819578    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819592    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819604    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-web-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819615    2560 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-cluster-tls-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819625    2560 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/181aadd6-ba89-4852-88ad-95450c8c284f-metrics-client-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819636    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-main-tls\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819647    2560 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/181aadd6-ba89-4852-88ad-95450c8c284f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819658    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/181aadd6-ba89-4852-88ad-95450c8c284f-config-out\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:49.819823 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:49.819669    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/181aadd6-ba89-4852-88ad-95450c8c284f-tls-assets\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:50.533456 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.533423    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"181aadd6-ba89-4852-88ad-95450c8c284f","Type":"ContainerDied","Data":"dc074aead4eea0d2387254fa4aead6616dd9a453dc1aedeb6c90017c23d5f60e"}
Jul 18 01:56:50.533920 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.533500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.533920 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.533497    2560 scope.go:117] "RemoveContainer" containerID="685906cd9427f71692f405a2a7b2e26c1bd44490ef2d2634142e53b44cd5a262"
Jul 18 01:56:50.541759 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.541739    2560 scope.go:117] "RemoveContainer" containerID="cbe9f3509f4e2b7c79443703df09a36abb8331a11cf9eca17014733991b11b63"
Jul 18 01:56:50.550314 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.550297    2560 scope.go:117] "RemoveContainer" containerID="cd4ababa50540f11186bf2fa6dafda97be19601a856d64e2f5008128de60b0c4"
Jul 18 01:56:50.556900 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.556872    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:50.561702 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.559508    2560 scope.go:117] "RemoveContainer" containerID="061ea66e3afa36bd0ae3d1425b00fd243ee16f0d1cd19d94c48aa5e659848b2a"
Jul 18 01:56:50.562649 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.562623    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:50.569380 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.569356    2560 scope.go:117] "RemoveContainer" containerID="cbe946e029b87c45e1b7a65d651a85d6fc226c407656616ee0d039498ab9e58c"
Jul 18 01:56:50.576791 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.576774    2560 scope.go:117] "RemoveContainer" containerID="af4931f018297bc1cdaa43f268b779404591fe983c40deaa2123fd6781fdaab6"
Jul 18 01:56:50.584413 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.584396    2560 scope.go:117] "RemoveContainer" containerID="67b78639fda0a9c03b8aef7db0fcd9e95100e6bed2b8b01b51cb371a5466b0a4"
Jul 18 01:56:50.590058 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590038    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:50.590372 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590361    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerName="registry"
Jul 18 01:56:50.590409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590375    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerName="registry"
Jul 18 01:56:50.590409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590383    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-web"
Jul 18 01:56:50.590409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590388    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-web"
Jul 18 01:56:50.590409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590398    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="prom-label-proxy"
Jul 18 01:56:50.590409 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590404    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="prom-label-proxy"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590411    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="init-config-reloader"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590417    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="init-config-reloader"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590422    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="alertmanager"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590427    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="alertmanager"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590434    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590439    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590444    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a00c655a-c222-4203-b371-5da034253e34" containerName="console"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590449    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a00c655a-c222-4203-b371-5da034253e34" containerName="console"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590458    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="config-reloader"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590463    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="config-reloader"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590470    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-metric"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590474    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-metric"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590482    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" containerName="console"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590487    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" containerName="console"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590553    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590565    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="alertmanager"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590571    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a10a0f1c-7ae9-42aa-8422-751679b17d8f" containerName="registry"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590576    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="prom-label-proxy"
Jul 18 01:56:50.590574 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590582    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a00c655a-c222-4203-b371-5da034253e34" containerName="console"
Jul 18 01:56:50.591182 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590588    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-metric"
Jul 18 01:56:50.591182 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590593    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac7b291-6fd7-40b0-95f0-f5430a238ba0" containerName="console"
Jul 18 01:56:50.591182 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590600    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="config-reloader"
Jul 18 01:56:50.591182 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.590605    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" containerName="kube-rbac-proxy-web"
Jul 18 01:56:50.597224 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.597204    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.599670 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599653    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 18 01:56:50.599776 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599751    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 18 01:56:50.599834 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599777    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 18 01:56:50.599893 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599878    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-zvlfn\""
Jul 18 01:56:50.599947 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599931    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 18 01:56:50.600001 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.599960    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 18 01:56:50.600116 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.600092    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 18 01:56:50.600186 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.600105    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 18 01:56:50.600687 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.600669    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 18 01:56:50.606159 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.606137    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 18 01:56:50.607340 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.607318    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:50.624326 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624302    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624348    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624450 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624366    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624597 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624449    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624597 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624478    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-config-out\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624597 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-config-volume\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624602    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624645    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624713 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624690    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624809 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624719    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-web-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624809 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624734    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.624809 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.624749    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82zc2\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-kube-api-access-82zc2\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726145 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726040    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726145 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726098    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726145 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726138    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726173    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-web-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726194    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726213    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82zc2\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-kube-api-access-82zc2\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726243    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726265    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726283    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726306    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726342    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726371    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-config-out\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.726423 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.726406    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-config-volume\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.727303 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.727215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.727422 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.727367    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.728334 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.728281    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f47886-ac01-4a5a-b753-82f753ac2623-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.729580 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.729555    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-config-volume\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.729655 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.729636    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-tls-assets\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.729707 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.729675    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.729836 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.729813    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.730066 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.730048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.730349 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.730330    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.730640 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.730618    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/08f47886-ac01-4a5a-b753-82f753ac2623-config-out\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.730763 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.730746    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-web-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.731447 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.731427    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/08f47886-ac01-4a5a-b753-82f753ac2623-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.733947 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.733929    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82zc2\" (UniqueName: \"kubernetes.io/projected/08f47886-ac01-4a5a-b753-82f753ac2623-kube-api-access-82zc2\") pod \"alertmanager-main-0\" (UID: \"08f47886-ac01-4a5a-b753-82f753ac2623\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:50.907590 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:50.907553    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 18 01:56:51.050401 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:51.050373    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 18 01:56:51.051912 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:56:51.051883    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f47886_ac01_4a5a_b753_82f753ac2623.slice/crio-2b610b26d1ad13e695709e734e01ef14f3b6fd2de218403c5d55a35d722e9c4f WatchSource:0}: Error finding container 2b610b26d1ad13e695709e734e01ef14f3b6fd2de218403c5d55a35d722e9c4f: Status 404 returned error can't find the container with id 2b610b26d1ad13e695709e734e01ef14f3b6fd2de218403c5d55a35d722e9c4f
Jul 18 01:56:51.115083 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:51.115059    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="181aadd6-ba89-4852-88ad-95450c8c284f" path="/var/lib/kubelet/pods/181aadd6-ba89-4852-88ad-95450c8c284f/volumes"
Jul 18 01:56:51.543544 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:51.543485    2560 generic.go:356] "Generic (PLEG): container finished" podID="08f47886-ac01-4a5a-b753-82f753ac2623" containerID="d0026388060e13c14614e5295f32acc3803cebaf6402f7545a820a80458befc2" exitCode=0
Jul 18 01:56:51.543908 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:51.543571    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerDied","Data":"d0026388060e13c14614e5295f32acc3803cebaf6402f7545a820a80458befc2"}
Jul 18 01:56:51.543908 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:51.543639    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"2b610b26d1ad13e695709e734e01ef14f3b6fd2de218403c5d55a35d722e9c4f"}
Jul 18 01:56:52.551046 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551007    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"e70840ba0a7545b10038dd8b8861187ff54d9ac905d4c55f65fc1f71a410592c"}
Jul 18 01:56:52.551449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551110    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"e3d8a9a9cb7ab6c0624b1d18923c86d096f7f4ba25bf9f92c14716f74316f5c2"}
Jul 18 01:56:52.551449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551139    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"9a4eb0453010b364199566abbb1781f774e207441a1174af924d4bceed2983db"}
Jul 18 01:56:52.551449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551169    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"4cd2154170e4f027e72271091c0f66a273cf620e626c1aaece39d0d4569c1130"}
Jul 18 01:56:52.551449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551192    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"41b0fd0ba3d89edbc680a95908471bd583609e99ffe3c2f736f72ac7e6bee646"}
Jul 18 01:56:52.551449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.551225    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"08f47886-ac01-4a5a-b753-82f753ac2623","Type":"ContainerStarted","Data":"3da0cab1ee462062550d7949114def1ed3bb4aac9e1a37eff69a23bb4976aa03"}
Jul 18 01:56:52.582752 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.582699    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.564668944 podStartE2EDuration="2.582681562s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.012618ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:52.580587162 +0000 UTC m=+381.902967327" watchObservedRunningTime="2026-07-18 01:56:52.582681562 +0000 UTC m=+381.905061728"
Jul 18 01:56:52.605419 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.605381    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.605846    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="prometheus" containerID="cri-o://110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76" gracePeriod=600
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.605860    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-thanos" containerID="cri-o://732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7" gracePeriod=600
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.605940    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-web" containerID="cri-o://b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479" gracePeriod=600
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.606040    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="thanos-sidecar" containerID="cri-o://49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70" gracePeriod=600
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.606084    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="config-reloader" containerID="cri-o://45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2" gracePeriod=600
Jul 18 01:56:52.606508 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.606035    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy" containerID="cri-o://9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf" gracePeriod=600
Jul 18 01:56:52.878157 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.878130    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:52.945961 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.945924    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94n7l\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.945975    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.945999    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946019    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946039    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946064    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946099    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946122    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946171    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946218    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946257    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946287    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946313    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946344    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946343    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.946538 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946480    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.946938 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.946761    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.947260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947173    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.947539 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947353    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.947539 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947415    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947539 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947449    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947539 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947483    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947539 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947533    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947807 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947591    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947807 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947620    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle\") pod \"53b02bb1-48bd-4d06-8364-19f4902ef341\" (UID: \"53b02bb1-48bd-4d06-8364-19f4902ef341\") "
Jul 18 01:56:52.947909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947861    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:52.947909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947873    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:52.947909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947882    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:52.947909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947892    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:52.947909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.947901    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:52.949592 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.948366    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.949592 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.949254    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:56:52.949787 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.949616    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:52.950502 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.950470    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.952306 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.952273    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.952909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.952775    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config" (OuterVolumeSpecName: "config") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.952909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.952862    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l" (OuterVolumeSpecName: "kube-api-access-94n7l") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "kube-api-access-94n7l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:52.952909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.952875    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out" (OuterVolumeSpecName: "config-out") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 01:56:52.953343 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.953319    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.953495 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.953461    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.953747 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.953726    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.954951 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.954927    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.955609 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.955580    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:52.955699 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.955607    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:56:52.966359 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:52.966322    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config" (OuterVolumeSpecName: "web-config") pod "53b02bb1-48bd-4d06-8364-19f4902ef341" (UID: "53b02bb1-48bd-4d06-8364-19f4902ef341"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:56:53.049037 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.048999    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049037 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049029    2560 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-metrics-client-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049037 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049042    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049051    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049061    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94n7l\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-kube-api-access-94n7l\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049070    2560 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-web-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049078    2560 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-grpc-tls\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049085    2560 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049093    2560 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-prometheus-k8s-db\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049101    2560 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/53b02bb1-48bd-4d06-8364-19f4902ef341-config-out\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049108    2560 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/53b02bb1-48bd-4d06-8364-19f4902ef341-tls-assets\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049115    2560 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/53b02bb1-48bd-4d06-8364-19f4902ef341-configmap-metrics-client-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049124    2560 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049133    2560 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-secret-kube-rbac-proxy\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.049265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.049142    2560 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/53b02bb1-48bd-4d06-8364-19f4902ef341-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:56:53.558854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558825    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7" exitCode=0
Jul 18 01:56:53.558854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558848    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf" exitCode=0
Jul 18 01:56:53.558854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558854    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479" exitCode=0
Jul 18 01:56:53.558854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558860    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70" exitCode=0
Jul 18 01:56:53.558854 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558864    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2" exitCode=0
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558869    2560 generic.go:356] "Generic (PLEG): container finished" podID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76" exitCode=0
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558901    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558926    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.558999    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559026    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559053    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559066    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559076    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559101    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"}
Jul 18 01:56:53.559403 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.559124    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"53b02bb1-48bd-4d06-8364-19f4902ef341","Type":"ContainerDied","Data":"55e2860d12fdfc4183d35769a39530fee6bf9b4d29e75ddcbf27ec716ab03b4b"}
Jul 18 01:56:53.567410 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.567392    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.575313 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.575296    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.582536 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.582487    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:53.584436 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.584414    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.586081 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.586060    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:53.591893 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.591874    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.599302 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.599281    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.607627 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.607458    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.613109 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613084    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:53.613402 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613391    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="thanos-sidecar"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613404    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="thanos-sidecar"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613416    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="init-config-reloader"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613422    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="init-config-reloader"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613428    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="config-reloader"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613433    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="config-reloader"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613442    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-web"
Jul 18 01:56:53.613453 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613448    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-web"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613458    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="prometheus"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613463    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="prometheus"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613471    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613476    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613483    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-thanos"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613489    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-thanos"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613559    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="thanos-sidecar"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613568    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-thanos"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613575    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="config-reloader"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613581    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613587    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="prometheus"
Jul 18 01:56:53.613665 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.613592    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" containerName="kube-rbac-proxy-web"
Jul 18 01:56:53.616148 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.616125    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.616481 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.616460    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.616563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.616490    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.616563 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.616509    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.616822 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.616802    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.616894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.616833    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.616894 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.616857    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.617109 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.617092    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.617147 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617116    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.617147 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617132    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.617394 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.617375    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.617440 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617401    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.617440 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617417    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.617641 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.617623    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.617697 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617645    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.617697 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617657    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.617883 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.617862    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.617924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617891    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.617924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.617914    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.618127 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:56:53.618111    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.618165 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618130    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.618165 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618142    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.618323 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618306    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.618373 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618323    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.618565 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618546    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.618615 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618566    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.618794 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618765    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.618794 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618792    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.618934 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.618921    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.619032 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619015    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.619088 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619033    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.619578 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619549    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.619578 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619574    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.619848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619830    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.619932 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.619848    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.620100 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620082    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.620178 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620103    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.620337 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620319    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.620383 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620339    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.620632 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620605    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.620704 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620632    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.620856 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620838    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.620928 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.620857    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.621070 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621051    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.621129 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621072    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.621286 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621265    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 18 01:56:53.621351 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621303    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.621351 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621323    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.621611 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621465    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 18 01:56:53.621611 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621542    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 18 01:56:53.621772 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621650    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 18 01:56:53.621772 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621652    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.621772 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621677    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.621772 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621683    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 18 01:56:53.621772 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621723    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 18 01:56:53.622017 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.621813    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 18 01:56:53.622118 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622098    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 18 01:56:53.622177 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622123    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 18 01:56:53.622227 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622173    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-293gs02thg94l\""
Jul 18 01:56:53.622227 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622139    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 18 01:56:53.622344 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622322    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.622344 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622346    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.622465 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622381    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 18 01:56:53.622836 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622808    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.622836 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622833    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 18 01:56:53.622998 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622835    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.623057 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.622897    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zzr28\""
Jul 18 01:56:53.623389 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.623370    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.623389 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.623389    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.623715 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.623695    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.623804 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.623715    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.624541    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.624571    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626253    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626287    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626570    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626596    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626866    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.626890    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627169    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627191    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627461    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627479    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627940    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.627958    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.628264    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.628411    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.628501 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.628432    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.629631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629069    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.629631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629097    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.629631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629348    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.629631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629371    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.629840 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629641    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.629840 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629666    2560 scope.go:117] "RemoveContainer" containerID="732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"
Jul 18 01:56:53.629969 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629945    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7"} err="failed to get container status \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": rpc error: code = NotFound desc = could not find container \"732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7\": container with ID starting with 732acf05a5fb34999a7af122aa34fc9d3c44f6b5e0cf2ef7cf98822e58dd38f7 not found: ID does not exist"
Jul 18 01:56:53.629969 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.629969    2560 scope.go:117] "RemoveContainer" containerID="9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"
Jul 18 01:56:53.632265 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632235    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf"} err="failed to get container status \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": rpc error: code = NotFound desc = could not find container \"9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf\": container with ID starting with 9bea3a560c87e5000e6c1e0eec46bf8afc4255c71b5897eabbdc375a19e392cf not found: ID does not exist"
Jul 18 01:56:53.632571 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632269    2560 scope.go:117] "RemoveContainer" containerID="b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"
Jul 18 01:56:53.632661 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632590    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479"} err="failed to get container status \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": rpc error: code = NotFound desc = could not find container \"b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479\": container with ID starting with b5adedf1f0d31c0d5f177feeee1f9512118a297e9e181e751ec49dcf2ecea479 not found: ID does not exist"
Jul 18 01:56:53.632661 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632637    2560 scope.go:117] "RemoveContainer" containerID="49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"
Jul 18 01:56:53.632965 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632941    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70"} err="failed to get container status \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": rpc error: code = NotFound desc = could not find container \"49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70\": container with ID starting with 49549e04d56c15f8ae5f74e70cc5ec7f827701db14d273dee04604b71b154e70 not found: ID does not exist"
Jul 18 01:56:53.633038 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.632966    2560 scope.go:117] "RemoveContainer" containerID="45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"
Jul 18 01:56:53.633338 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.633305    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2"} err="failed to get container status \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": rpc error: code = NotFound desc = could not find container \"45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2\": container with ID starting with 45e2f36251e576462f8af2d7f30fea3965b8df7449a569dcd26612ef536979b2 not found: ID does not exist"
Jul 18 01:56:53.633449 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.633341    2560 scope.go:117] "RemoveContainer" containerID="110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"
Jul 18 01:56:53.633503 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.633463    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 18 01:56:53.634598 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.634570    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76"} err="failed to get container status \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": rpc error: code = NotFound desc = could not find container \"110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76\": container with ID starting with 110d9f3db073044837ca4d5b425ffe7b52e1b7e5dbdd9f3be141ca1812ef8b76 not found: ID does not exist"
Jul 18 01:56:53.634691 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.634599    2560 scope.go:117] "RemoveContainer" containerID="820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"
Jul 18 01:56:53.634907 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.634885    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18"} err="failed to get container status \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": rpc error: code = NotFound desc = could not find container \"820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18\": container with ID starting with 820c5270e61042f4fae4a5eec7f4ece89eaaf295fd1c44ae5b41bd7559f24e18 not found: ID does not exist"
Jul 18 01:56:53.635225 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.635211    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:53.654849 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.654816    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.654857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.654892    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.654918    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.654941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49q6h\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-kube-api-access-49q6h\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655016 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655006    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655064    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-web-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655148    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655262    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655444    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655477    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655547    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-config-out\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655575    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.655591 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655595    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.656011 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655627    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.656011 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.656011 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655689    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.656011 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655739    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.656011 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.655767    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757039    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757076    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757103    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-config-out\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757120    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757417 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757134    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757417 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757251    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757417 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757312    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757417 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757363    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757667 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757643    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757731 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757391    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757778 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757751    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757829 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757795    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757886 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757827    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.757948 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.757857    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758041    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758060 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758048    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758071    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49q6h\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-kube-api-access-49q6h\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758108    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758174 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758169    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-web-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758365 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758200    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758365 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758236    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.758365 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.758305    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.760283 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.760256    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.760283 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.760277    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/43839636-be49-4df1-8ccf-5f1eec551331-config-out\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.761035 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.761004    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.761151 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.761130    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.761210 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.761169    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762329 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.761720    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762329 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.761782    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762329 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.762027    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762329 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.762028    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762329 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.762262    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762837 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.762573    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.762837 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.762591    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/43839636-be49-4df1-8ccf-5f1eec551331-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.763086 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.763059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.763623 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.763595    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-web-config\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.764115 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.764094    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/43839636-be49-4df1-8ccf-5f1eec551331-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.764330 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.764313    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.767918 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.767897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49q6h\" (UniqueName: \"kubernetes.io/projected/43839636-be49-4df1-8ccf-5f1eec551331-kube-api-access-49q6h\") pod \"prometheus-k8s-0\" (UID: \"43839636-be49-4df1-8ccf-5f1eec551331\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:53.935008 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:53.934952    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:56:54.075473 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:54.075444    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 18 01:56:54.077022 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:56:54.076997    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43839636_be49_4df1_8ccf_5f1eec551331.slice/crio-282064e92b06b56835c11841a7168139d5fb2272489d64b5896aff427b62da2b WatchSource:0}: Error finding container 282064e92b06b56835c11841a7168139d5fb2272489d64b5896aff427b62da2b: Status 404 returned error can't find the container with id 282064e92b06b56835c11841a7168139d5fb2272489d64b5896aff427b62da2b
Jul 18 01:56:54.564865 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:54.564831    2560 generic.go:356] "Generic (PLEG): container finished" podID="43839636-be49-4df1-8ccf-5f1eec551331" containerID="dc5467c6cf17bdd8982594cc74afab2d4979c6324ac2baf406f22328a2426653" exitCode=0
Jul 18 01:56:54.565295 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:54.564919    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerDied","Data":"dc5467c6cf17bdd8982594cc74afab2d4979c6324ac2baf406f22328a2426653"}
Jul 18 01:56:54.565295 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:54.564993    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"282064e92b06b56835c11841a7168139d5fb2272489d64b5896aff427b62da2b"}
Jul 18 01:56:55.115768 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.115689    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53b02bb1-48bd-4d06-8364-19f4902ef341" path="/var/lib/kubelet/pods/53b02bb1-48bd-4d06-8364-19f4902ef341/volumes"
Jul 18 01:56:55.574507 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"85ff85906854729c7a2967ed7dd7fd40a306b297db3ead921cf966ab1747cb18"}
Jul 18 01:56:55.574879 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574573    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"1bcf229b10046f90e52e15eb8826d55ec5c0ff96faf8f2e44a5e11e2ca074f61"}
Jul 18 01:56:55.574879 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574623    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"e58006c15c2bed38fc6578e093b8a43c65fb903b113ed7706bb67596be7bbeba"}
Jul 18 01:56:55.574879 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574659    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"9d1ba51ace91ac257e47981c45e860750cb17b371a734d50b4ecc26a26f23739"}
Jul 18 01:56:55.574879 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574696    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"73f6a644ac40a35e1aa0f5375ab25878f61cdfd03ce245c3c0aa2ce5e02a4f79"}
Jul 18 01:56:55.574879 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.574731    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"43839636-be49-4df1-8ccf-5f1eec551331","Type":"ContainerStarted","Data":"794a1a30bf2598051ef3ef2d1703504c7ff9a75d96abcb52d96f26f7a7a96b81"}
Jul 18 01:56:55.604767 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:55.604709    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.582167777 podStartE2EDuration="2.604692318s" totalImagesPullingTime="0s" totalInitContainerRuntime="22.524541ms" isStatefulPod=false podCreationTimestamp="2026-07-18 01:56:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:56:55.602105698 +0000 UTC m=+384.924485887" watchObservedRunningTime="2026-07-18 01:56:55.604692318 +0000 UTC m=+384.927072569"
Jul 18 01:56:58.935280 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:56:58.935225    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:57:03.334419 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.334374    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 01:57:03.339723 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.339696    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.352455 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.352426    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 01:57:03.445371 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445371 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445414    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445475    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445503    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jn65\" (UniqueName: \"kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.445619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.445593    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546550 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546760 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546578    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546760 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546604    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546760 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546637    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jn65\" (UniqueName: \"kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546760 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546663    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.546760 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546732    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.547008 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.546752    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.547298 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.547269    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.547408 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.547274    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.547641 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.547610    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.547811 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.547789    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.549172 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.549154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.549419 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.549399    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.554490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.554465    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jn65\" (UniqueName: \"kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65\") pod \"console-6f4bfd848c-c2k5b\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") " pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.649785 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.649745    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:03.786218 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:03.786190    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 01:57:03.788934 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:57:03.788901    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod058eda78_baae_4b89_a4ee_c039186171ae.slice/crio-a5fb66288f108c8821e0f437380eb7136632fef49d955f22b2f9bc1b873250fc WatchSource:0}: Error finding container a5fb66288f108c8821e0f437380eb7136632fef49d955f22b2f9bc1b873250fc: Status 404 returned error can't find the container with id a5fb66288f108c8821e0f437380eb7136632fef49d955f22b2f9bc1b873250fc
Jul 18 01:57:04.617434 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:04.617391    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4bfd848c-c2k5b" event={"ID":"058eda78-baae-4b89-a4ee-c039186171ae","Type":"ContainerStarted","Data":"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"}
Jul 18 01:57:04.617827 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:04.617455    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4bfd848c-c2k5b" event={"ID":"058eda78-baae-4b89-a4ee-c039186171ae","Type":"ContainerStarted","Data":"a5fb66288f108c8821e0f437380eb7136632fef49d955f22b2f9bc1b873250fc"}
Jul 18 01:57:04.634015 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:04.633967    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f4bfd848c-c2k5b" podStartSLOduration=1.633950169 podStartE2EDuration="1.633950169s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:57:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:57:04.632701747 +0000 UTC m=+393.955081926" watchObservedRunningTime="2026-07-18 01:57:04.633950169 +0000 UTC m=+393.956330392"
Jul 18 01:57:13.650651 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:13.650606    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:13.651071 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:13.650682    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:13.655832 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:13.655809    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:13.661946 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:13.661921    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 01:57:13.731274 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:13.731243    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:57:38.754919 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:38.754880    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67796f69fd-dhpjh" podUID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" containerName="console" containerID="cri-o://94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c" gracePeriod=15
Jul 18 01:57:39.013290 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.013221    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:57:39.063909 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.063876    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.063926    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.063995    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064022    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkbtq\" (UniqueName: \"kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064082 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064046    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064114    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064275 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064141    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config\") pod \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\" (UID: \"e535e150-d0e8-4f56-a83f-aee5fba9df5b\") "
Jul 18 01:57:39.064379 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064313    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca" (OuterVolumeSpecName: "service-ca") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:57:39.064562 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064535    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:57:39.064743 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064566    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.064743 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064569    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:57:39.064743 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.064614    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config" (OuterVolumeSpecName: "console-config") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 01:57:39.066460 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.066440    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:57:39.066779 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.066760    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 01:57:39.066839 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.066771    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq" (OuterVolumeSpecName: "kube-api-access-zkbtq") pod "e535e150-d0e8-4f56-a83f-aee5fba9df5b" (UID: "e535e150-d0e8-4f56-a83f-aee5fba9df5b"). InnerVolumeSpecName "kube-api-access-zkbtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 01:57:39.165378 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165340    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-trusted-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.165378 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165373    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkbtq\" (UniqueName: \"kubernetes.io/projected/e535e150-d0e8-4f56-a83f-aee5fba9df5b-kube-api-access-zkbtq\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.165378 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165384    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.165647 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165392    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-oauth-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.165647 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165403    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.165647 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.165412    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e535e150-d0e8-4f56-a83f-aee5fba9df5b-console-oauth-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 01:57:39.773758 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.773719    2560 generic.go:356] "Generic (PLEG): container finished" podID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" containerID="94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c" exitCode=0
Jul 18 01:57:39.774235 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.773782    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67796f69fd-dhpjh"
Jul 18 01:57:39.774235 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.773807    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67796f69fd-dhpjh" event={"ID":"e535e150-d0e8-4f56-a83f-aee5fba9df5b","Type":"ContainerDied","Data":"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"}
Jul 18 01:57:39.774235 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.773873    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67796f69fd-dhpjh" event={"ID":"e535e150-d0e8-4f56-a83f-aee5fba9df5b","Type":"ContainerDied","Data":"22d8788e13f62dd45b05830eeb1909668a366f3370751a4ae7c8616e802796ce"}
Jul 18 01:57:39.774235 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.773895    2560 scope.go:117] "RemoveContainer" containerID="94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"
Jul 18 01:57:39.782380 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.782355    2560 scope.go:117] "RemoveContainer" containerID="94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"
Jul 18 01:57:39.782656 ip-10-0-129-141 kubenswrapper[2560]: E0718 01:57:39.782635    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c\": container with ID starting with 94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c not found: ID does not exist" containerID="94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"
Jul 18 01:57:39.782732 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.782668    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c"} err="failed to get container status \"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c\": rpc error: code = NotFound desc = could not find container \"94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c\": container with ID starting with 94a3e68e948fb9ebc4a4203ad37eb8b8638733bd0a83f90bbce2ec062eb37e1c not found: ID does not exist"
Jul 18 01:57:39.789637 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.789604    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:57:39.791835 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:39.791811    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67796f69fd-dhpjh"]
Jul 18 01:57:41.115956 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:41.115926    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" path="/var/lib/kubelet/pods/e535e150-d0e8-4f56-a83f-aee5fba9df5b/volumes"
Jul 18 01:57:53.935593 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:53.935483    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:57:53.951680 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:53.951654    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:57:54.859218 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:57:54.859192    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 18 01:58:46.552368 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.552329    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"]
Jul 18 01:58:46.552803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.552699    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" containerName="console"
Jul 18 01:58:46.552803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.552711    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" containerName="console"
Jul 18 01:58:46.552803 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.552779    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e535e150-d0e8-4f56-a83f-aee5fba9df5b" containerName="console"
Jul 18 01:58:46.556828 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.556809    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.559309 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.559285    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-l48qq\""
Jul 18 01:58:46.559309 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.559285    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 18 01:58:46.560160 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.560145    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 18 01:58:46.563677 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.563650    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"]
Jul 18 01:58:46.651642 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.651607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrss\" (UniqueName: \"kubernetes.io/projected/c2ca91c3-0098-47eb-847c-47b82c3be8b9-kube-api-access-bjrss\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.651811 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.651664    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2ca91c3-0098-47eb-847c-47b82c3be8b9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.752494 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.752453    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2ca91c3-0098-47eb-847c-47b82c3be8b9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.752682 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.752568    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrss\" (UniqueName: \"kubernetes.io/projected/c2ca91c3-0098-47eb-847c-47b82c3be8b9-kube-api-access-bjrss\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.752857 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.752838    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2ca91c3-0098-47eb-847c-47b82c3be8b9-tmp\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.760572 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.760550    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrss\" (UniqueName: \"kubernetes.io/projected/c2ca91c3-0098-47eb-847c-47b82c3be8b9-kube-api-access-bjrss\") pod \"openshift-lws-operator-fd8ccff4c-4756v\" (UID: \"c2ca91c3-0098-47eb-847c-47b82c3be8b9\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:46.882874 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:46.882842    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"
Jul 18 01:58:47.015374 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:47.015355    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v"]
Jul 18 01:58:47.018567 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:58:47.018510    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2ca91c3_0098_47eb_847c_47b82c3be8b9.slice/crio-505a41e23b15c00bda221d0e956ce6cbcde6802a44e6b62441336f036813619c WatchSource:0}: Error finding container 505a41e23b15c00bda221d0e956ce6cbcde6802a44e6b62441336f036813619c: Status 404 returned error can't find the container with id 505a41e23b15c00bda221d0e956ce6cbcde6802a44e6b62441336f036813619c
Jul 18 01:58:47.054667 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:47.054631    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v" event={"ID":"c2ca91c3-0098-47eb-847c-47b82c3be8b9","Type":"ContainerStarted","Data":"505a41e23b15c00bda221d0e956ce6cbcde6802a44e6b62441336f036813619c"}
Jul 18 01:58:50.071749 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:50.071703    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v" event={"ID":"c2ca91c3-0098-47eb-847c-47b82c3be8b9","Type":"ContainerStarted","Data":"4d259346aec078d97dfdcf4fb478fedeac5a4234e0c7eb165c3dc3d8bb8ceaea"}
Jul 18 01:58:50.088047 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:58:50.087992    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-4756v" podStartSLOduration=1.8286753340000002 podStartE2EDuration="4.087973997s" totalImagesPullingTime="2.259298663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:58:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:58:50.086580258 +0000 UTC m=+499.408960427" watchObservedRunningTime="2026-07-18 01:58:50.087973997 +0000 UTC m=+499.410354162"
Jul 18 01:59:05.233731 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.233697    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"]
Jul 18 01:59:05.238653 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.238631    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.242848 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.242822    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 18 01:59:05.243986 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.243961    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 18 01:59:05.244104 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.243970    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 18 01:59:05.246025 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.245940    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-sdk5z\""
Jul 18 01:59:05.246176 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.246155    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"]
Jul 18 01:59:05.325083 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.325040    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-metrics-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.325260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.325096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e289052c-c4d3-4c41-9eb5-ee038b41d038-manager-config\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.325260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.325199    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtkjh\" (UniqueName: \"kubernetes.io/projected/e289052c-c4d3-4c41-9eb5-ee038b41d038-kube-api-access-qtkjh\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.325260 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.325236    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.426311 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.426271    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-metrics-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.426643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.426332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e289052c-c4d3-4c41-9eb5-ee038b41d038-manager-config\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.426643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.426372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtkjh\" (UniqueName: \"kubernetes.io/projected/e289052c-c4d3-4c41-9eb5-ee038b41d038-kube-api-access-qtkjh\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.426643 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.426395    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.429907 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.427938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e289052c-c4d3-4c41-9eb5-ee038b41d038-manager-config\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.429907 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.429834    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-metrics-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.432903 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.430593    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e289052c-c4d3-4c41-9eb5-ee038b41d038-cert\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.439507 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.439486    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtkjh\" (UniqueName: \"kubernetes.io/projected/e289052c-c4d3-4c41-9eb5-ee038b41d038-kube-api-access-qtkjh\") pod \"lws-controller-manager-64d778cd56-c8szz\" (UID: \"e289052c-c4d3-4c41-9eb5-ee038b41d038\") " pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.549830 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.549727    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:05.705754 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:05.705724    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"]
Jul 18 01:59:05.708007 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:59:05.707974    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode289052c_c4d3_4c41_9eb5_ee038b41d038.slice/crio-17c715d92757c1d9189d2e44ebe095cfaa116a27c4d8c98ab5fa193b7d2e6966 WatchSource:0}: Error finding container 17c715d92757c1d9189d2e44ebe095cfaa116a27c4d8c98ab5fa193b7d2e6966: Status 404 returned error can't find the container with id 17c715d92757c1d9189d2e44ebe095cfaa116a27c4d8c98ab5fa193b7d2e6966
Jul 18 01:59:06.143737 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.143690    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz" event={"ID":"e289052c-c4d3-4c41-9eb5-ee038b41d038","Type":"ContainerStarted","Data":"17c715d92757c1d9189d2e44ebe095cfaa116a27c4d8c98ab5fa193b7d2e6966"}
Jul 18 01:59:06.279067 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.279032    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"]
Jul 18 01:59:06.282825 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.282802    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.286046 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.286019    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 18 01:59:06.286178 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.286076    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9f4g4\""
Jul 18 01:59:06.286419 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.286396    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 18 01:59:06.286513 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.286441    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 18 01:59:06.287134 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.287110    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 18 01:59:06.298619 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.298594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"]
Jul 18 01:59:06.333897 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.333864    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-manifests\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.333897 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.333903    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzhz8\" (UniqueName: \"kubernetes.io/projected/424e896d-b4b3-44ee-b24c-663b84849850-kube-api-access-zzhz8\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.334175 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.333978    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.334175 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.334068    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.334175 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.334145    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-tmp\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.434747 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.434651    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-tmp\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.434924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.434753    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-manifests\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.434924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.434782    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzhz8\" (UniqueName: \"kubernetes.io/projected/424e896d-b4b3-44ee-b24c-663b84849850-kube-api-access-zzhz8\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.434924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.434813    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.434924 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.434842    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.435285 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.435255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-manifests\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.437968 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.437830    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/424e896d-b4b3-44ee-b24c-663b84849850-tmp\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.438202 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.438172    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-webhook-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.438331 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.438255    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/424e896d-b4b3-44ee-b24c-663b84849850-apiservice-cert\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.443717 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.443689    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzhz8\" (UniqueName: \"kubernetes.io/projected/424e896d-b4b3-44ee-b24c-663b84849850-kube-api-access-zzhz8\") pod \"opendatahub-operator-controller-manager-75cddc5ddd-7zg4j\" (UID: \"424e896d-b4b3-44ee-b24c-663b84849850\") " pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:06.596478 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:06.596439    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:07.019490 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:07.019457    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"]
Jul 18 01:59:07.022770 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:59:07.022741    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod424e896d_b4b3_44ee_b24c_663b84849850.slice/crio-090d920ac995d6ec232cdbcb09df6b220ffe991667188c9021ba15ebde4b75d6 WatchSource:0}: Error finding container 090d920ac995d6ec232cdbcb09df6b220ffe991667188c9021ba15ebde4b75d6: Status 404 returned error can't find the container with id 090d920ac995d6ec232cdbcb09df6b220ffe991667188c9021ba15ebde4b75d6
Jul 18 01:59:07.149424 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:07.149385    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz" event={"ID":"e289052c-c4d3-4c41-9eb5-ee038b41d038","Type":"ContainerStarted","Data":"77b3407af43220b94f8d09fe62d0f9f55a712e69afa116f2d851f256f415b6ea"}
Jul 18 01:59:07.149672 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:07.149451    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:07.150611 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:07.150587    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j" event={"ID":"424e896d-b4b3-44ee-b24c-663b84849850","Type":"ContainerStarted","Data":"090d920ac995d6ec232cdbcb09df6b220ffe991667188c9021ba15ebde4b75d6"}
Jul 18 01:59:07.176938 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:07.176889    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz" podStartSLOduration=0.940553336 podStartE2EDuration="2.176873317s" totalImagesPullingTime="1.236319981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:07.175427112 +0000 UTC m=+516.497807280" watchObservedRunningTime="2026-07-18 01:59:07.176873317 +0000 UTC m=+516.499253483"
Jul 18 01:59:10.169063 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:10.169030    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j" event={"ID":"424e896d-b4b3-44ee-b24c-663b84849850","Type":"ContainerStarted","Data":"6a7184c9fccacc215e29a89f33f97b5e0fb532dcdab7620318b6097ace73a8fe"}
Jul 18 01:59:11.174366 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:11.174330    2560 generic.go:356] "Generic (PLEG): container finished" podID="424e896d-b4b3-44ee-b24c-663b84849850" containerID="6a7184c9fccacc215e29a89f33f97b5e0fb532dcdab7620318b6097ace73a8fe" exitCode=0
Jul 18 01:59:11.174770 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:11.174410    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j" event={"ID":"424e896d-b4b3-44ee-b24c-663b84849850","Type":"ContainerDied","Data":"6a7184c9fccacc215e29a89f33f97b5e0fb532dcdab7620318b6097ace73a8fe"}
Jul 18 01:59:12.180770 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:12.180735    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j" event={"ID":"424e896d-b4b3-44ee-b24c-663b84849850","Type":"ContainerStarted","Data":"59ae6640087f611e62a87f050ac2e8df084fdd17057969a8ba5ef9f237e9a1ba"}
Jul 18 01:59:12.181191 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:12.180866    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:12.202372 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:12.202319    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j" podStartSLOduration=1.891146854 podStartE2EDuration="6.202296661s" totalImagesPullingTime="3.216100872s" totalInitContainerRuntime="1.095048935s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:12.200582436 +0000 UTC m=+521.522962605" watchObservedRunningTime="2026-07-18 01:59:12.202296661 +0000 UTC m=+521.524676829"
Jul 18 01:59:18.160023 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:18.159987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64d778cd56-c8szz"
Jul 18 01:59:23.188289 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:23.188195    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-75cddc5ddd-7zg4j"
Jul 18 01:59:25.841653 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.841616    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"]
Jul 18 01:59:25.845091 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.845064    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:25.847642 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.847610    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 18 01:59:25.847781 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.847610    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 18 01:59:25.847858 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.847806    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9hdxh\""
Jul 18 01:59:25.858127 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.858097    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"]
Jul 18 01:59:25.922708 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.922667    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98102292-5e2d-4e91-be08-ac0d6041a6f5-tmp\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:25.922708 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.922710    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26np4\" (UniqueName: \"kubernetes.io/projected/98102292-5e2d-4e91-be08-ac0d6041a6f5-kube-api-access-26np4\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:25.922932 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:25.922817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98102292-5e2d-4e91-be08-ac0d6041a6f5-tls-certs\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.024360 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.024319    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98102292-5e2d-4e91-be08-ac0d6041a6f5-tmp\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.024360 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.024358    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26np4\" (UniqueName: \"kubernetes.io/projected/98102292-5e2d-4e91-be08-ac0d6041a6f5-kube-api-access-26np4\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.024631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.024414    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98102292-5e2d-4e91-be08-ac0d6041a6f5-tls-certs\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.026880 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.026844    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98102292-5e2d-4e91-be08-ac0d6041a6f5-tmp\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.026999 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.026932    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/98102292-5e2d-4e91-be08-ac0d6041a6f5-tls-certs\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.033399 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.033373    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26np4\" (UniqueName: \"kubernetes.io/projected/98102292-5e2d-4e91-be08-ac0d6041a6f5-kube-api-access-26np4\") pod \"kube-auth-proxy-65cb7b95cb-mcbdj\" (UID: \"98102292-5e2d-4e91-be08-ac0d6041a6f5\") " pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.155186 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.155156    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"
Jul 18 01:59:26.297926 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:26.297902    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj"]
Jul 18 01:59:26.299893 ip-10-0-129-141 kubenswrapper[2560]: W0718 01:59:26.299868    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98102292_5e2d_4e91_be08_ac0d6041a6f5.slice/crio-f73491276bec11227ac037739c5a8dabbd7fe55744adb1bcad428be71e7bf72f WatchSource:0}: Error finding container f73491276bec11227ac037739c5a8dabbd7fe55744adb1bcad428be71e7bf72f: Status 404 returned error can't find the container with id f73491276bec11227ac037739c5a8dabbd7fe55744adb1bcad428be71e7bf72f
Jul 18 01:59:27.255861 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:27.255808    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj" event={"ID":"98102292-5e2d-4e91-be08-ac0d6041a6f5","Type":"ContainerStarted","Data":"f73491276bec11227ac037739c5a8dabbd7fe55744adb1bcad428be71e7bf72f"}
Jul 18 01:59:29.268631 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:29.268591    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj" event={"ID":"98102292-5e2d-4e91-be08-ac0d6041a6f5","Type":"ContainerStarted","Data":"1c6f992c72dd28b7ca320ab00420106a11c1548fea6f0602e0570b927e6ea690"}
Jul 18 01:59:29.284332 ip-10-0-129-141 kubenswrapper[2560]: I0718 01:59:29.284276    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-65cb7b95cb-mcbdj" podStartSLOduration=2.150780824 podStartE2EDuration="4.284260391s" totalImagesPullingTime="2.133479567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 01:59:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 01:59:29.283072722 +0000 UTC m=+538.605452929" watchObservedRunningTime="2026-07-18 01:59:29.284260391 +0000 UTC m=+538.606640557"
Jul 18 02:00:48.823323 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.823285    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-9556bff77-qtjkf"]
Jul 18 02:00:48.825698 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.825677    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.836386 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.836357    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9556bff77-qtjkf"]
Jul 18 02:00:48.889290 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-oauth-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889290 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889285    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-service-ca\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889556 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889556 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889369    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-trusted-ca-bundle\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889556 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889390    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bjd\" (UniqueName: \"kubernetes.io/projected/bc2d391e-6881-41fd-978b-b90eb0155969-kube-api-access-49bjd\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889556 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889411    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-console-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.889556 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.889445    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-oauth-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990735 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-oauth-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990755    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-oauth-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-service-ca\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990817    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990853    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-trusted-ca-bundle\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bjd\" (UniqueName: \"kubernetes.io/projected/bc2d391e-6881-41fd-978b-b90eb0155969-kube-api-access-49bjd\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.990952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.990917    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-console-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.991737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.991644    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-oauth-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.991737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.991676    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-console-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.991737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.991660    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-service-ca\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.991971 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.991843    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bc2d391e-6881-41fd-978b-b90eb0155969-trusted-ca-bundle\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.993410 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.993391    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-oauth-config\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:48.993495 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:48.993466    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bc2d391e-6881-41fd-978b-b90eb0155969-console-serving-cert\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:49.005929 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.005903    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bjd\" (UniqueName: \"kubernetes.io/projected/bc2d391e-6881-41fd-978b-b90eb0155969-kube-api-access-49bjd\") pod \"console-9556bff77-qtjkf\" (UID: \"bc2d391e-6881-41fd-978b-b90eb0155969\") " pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:49.136075 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.136043    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:49.276737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.276712    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9556bff77-qtjkf"]
Jul 18 02:00:49.278978 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:00:49.278948    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc2d391e_6881_41fd_978b_b90eb0155969.slice/crio-3d1b65c66cb89e3bc706b1b62f59ef363335df0775d1100fa1a25c56eea6d776 WatchSource:0}: Error finding container 3d1b65c66cb89e3bc706b1b62f59ef363335df0775d1100fa1a25c56eea6d776: Status 404 returned error can't find the container with id 3d1b65c66cb89e3bc706b1b62f59ef363335df0775d1100fa1a25c56eea6d776
Jul 18 02:00:49.629722 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.629691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9556bff77-qtjkf" event={"ID":"bc2d391e-6881-41fd-978b-b90eb0155969","Type":"ContainerStarted","Data":"405d674ad7db8b298d236e67a0c8a2cb3d3dea9537b9dd2bf2ba5e3b9ebc5792"}
Jul 18 02:00:49.629897 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.629738    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9556bff77-qtjkf" event={"ID":"bc2d391e-6881-41fd-978b-b90eb0155969","Type":"ContainerStarted","Data":"3d1b65c66cb89e3bc706b1b62f59ef363335df0775d1100fa1a25c56eea6d776"}
Jul 18 02:00:49.649390 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:49.649333    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-9556bff77-qtjkf" podStartSLOduration=1.649316155 podStartE2EDuration="1.649316155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:00:49.64711874 +0000 UTC m=+618.969498906" watchObservedRunningTime="2026-07-18 02:00:49.649316155 +0000 UTC m=+618.971696321"
Jul 18 02:00:59.136179 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:59.136130    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:59.136730 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:59.136210    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:59.141120 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:59.141093    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:59.679978 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:59.679950    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9556bff77-qtjkf"
Jul 18 02:00:59.739570 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:00:59.739506    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 02:01:24.761919 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:24.761875    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6f4bfd848c-c2k5b" podUID="058eda78-baae-4b89-a4ee-c039186171ae" containerName="console" containerID="cri-o://7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952" gracePeriod=15
Jul 18 02:01:25.026958 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.026883    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 02:01:25.125832 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.125795    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126017 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.125880    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126017 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.125989    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jn65\" (UniqueName: \"kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126061    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126188 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126188 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126160    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126296 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126189    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert\") pod \"058eda78-baae-4b89-a4ee-c039186171ae\" (UID: \"058eda78-baae-4b89-a4ee-c039186171ae\") "
Jul 18 02:01:25.126296 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126228    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:01:25.126296 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126260    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:01:25.126540 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126499    2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-trusted-ca-bundle\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.126620 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126544    2560 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.126689 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126659    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config" (OuterVolumeSpecName: "console-config") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:01:25.126751 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.126728    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:01:25.128479 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.128454    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:01:25.128601 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.128500    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:01:25.128601 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.128513    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65" (OuterVolumeSpecName: "kube-api-access-4jn65") pod "058eda78-baae-4b89-a4ee-c039186171ae" (UID: "058eda78-baae-4b89-a4ee-c039186171ae"). InnerVolumeSpecName "kube-api-access-4jn65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:01:25.227046 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.227008    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jn65\" (UniqueName: \"kubernetes.io/projected/058eda78-baae-4b89-a4ee-c039186171ae-kube-api-access-4jn65\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.227046 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.227039    2560 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.227046 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.227048    2560 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/058eda78-baae-4b89-a4ee-c039186171ae-console-oauth-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.227046 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.227058    2560 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-console-config\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.227315 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.227066    2560 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/058eda78-baae-4b89-a4ee-c039186171ae-oauth-serving-cert\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:01:25.796413 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.796379    2560 generic.go:356] "Generic (PLEG): container finished" podID="058eda78-baae-4b89-a4ee-c039186171ae" containerID="7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952" exitCode=0
Jul 18 02:01:25.796876 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.796428    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f4bfd848c-c2k5b"
Jul 18 02:01:25.796876 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.796469    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4bfd848c-c2k5b" event={"ID":"058eda78-baae-4b89-a4ee-c039186171ae","Type":"ContainerDied","Data":"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"}
Jul 18 02:01:25.796876 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.796539    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f4bfd848c-c2k5b" event={"ID":"058eda78-baae-4b89-a4ee-c039186171ae","Type":"ContainerDied","Data":"a5fb66288f108c8821e0f437380eb7136632fef49d955f22b2f9bc1b873250fc"}
Jul 18 02:01:25.796876 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.796567    2560 scope.go:117] "RemoveContainer" containerID="7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"
Jul 18 02:01:25.805796 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.805780    2560 scope.go:117] "RemoveContainer" containerID="7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"
Jul 18 02:01:25.806059 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:01:25.806043    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952\": container with ID starting with 7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952 not found: ID does not exist" containerID="7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"
Jul 18 02:01:25.806109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.806066    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952"} err="failed to get container status \"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952\": rpc error: code = NotFound desc = could not find container \"7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952\": container with ID starting with 7ca54992f36729b2261cb5ed07a34394be067b3d99db5b3060cbbd6b00f64952 not found: ID does not exist"
Jul 18 02:01:25.818444 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.818422    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 02:01:25.821870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:25.821847    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f4bfd848c-c2k5b"]
Jul 18 02:01:27.116689 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:01:27.116655    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="058eda78-baae-4b89-a4ee-c039186171ae" path="/var/lib/kubelet/pods/058eda78-baae-4b89-a4ee-c039186171ae/volumes"
Jul 18 02:02:11.977772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.977731    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:02:11.978305 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.978234    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="058eda78-baae-4b89-a4ee-c039186171ae" containerName="console"
Jul 18 02:02:11.978305 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.978253    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="058eda78-baae-4b89-a4ee-c039186171ae" containerName="console"
Jul 18 02:02:11.978426 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.978373    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="058eda78-baae-4b89-a4ee-c039186171ae" containerName="console"
Jul 18 02:02:11.981671 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.981648    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:11.983910 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.983883    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 18 02:02:11.984045 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.984025    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 18 02:02:11.984100 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.984071    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 18 02:02:11.984835 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.984817    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4h5xr\""
Jul 18 02:02:11.989874 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:11.989856    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:02:12.014368 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.014328    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:02:12.048037 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.047994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eef6c496-1b11-4022-9053-e972d7f94589-config-file\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.048224 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.048063    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdvlx\" (UniqueName: \"kubernetes.io/projected/eef6c496-1b11-4022-9053-e972d7f94589-kube-api-access-vdvlx\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.148736 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.148702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdvlx\" (UniqueName: \"kubernetes.io/projected/eef6c496-1b11-4022-9053-e972d7f94589-kube-api-access-vdvlx\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.148966 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.148773    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eef6c496-1b11-4022-9053-e972d7f94589-config-file\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.149479 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.149458    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/eef6c496-1b11-4022-9053-e972d7f94589-config-file\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.157094 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.157060    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdvlx\" (UniqueName: \"kubernetes.io/projected/eef6c496-1b11-4022-9053-e972d7f94589-kube-api-access-vdvlx\") pod \"limitador-limitador-54d6d988d5-m72zp\" (UID: \"eef6c496-1b11-4022-9053-e972d7f94589\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.293835 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.293728    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:12.431976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.431952    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:02:12.434645 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:02:12.434617    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeef6c496_1b11_4022_9053_e972d7f94589.slice/crio-dbef12a67881688608b7b9662e955abaa90a097e417f05d9d7f2eef7cec9c5f5 WatchSource:0}: Error finding container dbef12a67881688608b7b9662e955abaa90a097e417f05d9d7f2eef7cec9c5f5: Status 404 returned error can't find the container with id dbef12a67881688608b7b9662e955abaa90a097e417f05d9d7f2eef7cec9c5f5
Jul 18 02:02:12.436491 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.436473    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:02:12.556031 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.555952    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:12.560725 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.560708    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:12.569755 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.569721    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-586bp\""
Jul 18 02:02:12.572268 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.572244    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:12.652682 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.652640    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk48l\" (UniqueName: \"kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l\") pod \"authorino-75d556bd77-zskrf\" (UID: \"b9563bf1-e9b1-49fc-8544-1055d660e1e8\") " pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:12.753377 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.753332    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk48l\" (UniqueName: \"kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l\") pod \"authorino-75d556bd77-zskrf\" (UID: \"b9563bf1-e9b1-49fc-8544-1055d660e1e8\") " pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:12.761179 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.761145    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk48l\" (UniqueName: \"kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l\") pod \"authorino-75d556bd77-zskrf\" (UID: \"b9563bf1-e9b1-49fc-8544-1055d660e1e8\") " pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:12.878488 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:12.878392    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:13.014817 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:13.014788    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:13.017574 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:02:13.017547    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9563bf1_e9b1_49fc_8544_1055d660e1e8.slice/crio-47be73c88149a163a4771ec17f415b80aeb3f8f432c5f20641aa007c763c6be7 WatchSource:0}: Error finding container 47be73c88149a163a4771ec17f415b80aeb3f8f432c5f20641aa007c763c6be7: Status 404 returned error can't find the container with id 47be73c88149a163a4771ec17f415b80aeb3f8f432c5f20641aa007c763c6be7
Jul 18 02:02:13.043066 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:13.043028    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zskrf" event={"ID":"b9563bf1-e9b1-49fc-8544-1055d660e1e8","Type":"ContainerStarted","Data":"47be73c88149a163a4771ec17f415b80aeb3f8f432c5f20641aa007c763c6be7"}
Jul 18 02:02:13.044145 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:13.044121    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp" event={"ID":"eef6c496-1b11-4022-9053-e972d7f94589","Type":"ContainerStarted","Data":"dbef12a67881688608b7b9662e955abaa90a097e417f05d9d7f2eef7cec9c5f5"}
Jul 18 02:02:18.072495 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:18.072455    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zskrf" event={"ID":"b9563bf1-e9b1-49fc-8544-1055d660e1e8","Type":"ContainerStarted","Data":"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"}
Jul 18 02:02:18.074009 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:18.073984    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp" event={"ID":"eef6c496-1b11-4022-9053-e972d7f94589","Type":"ContainerStarted","Data":"448ec48877f9c977a59754f121947d3e250203db77aeb9e4b64786fa3afaf394"}
Jul 18 02:02:18.074140 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:18.074038    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:18.086593 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:18.086549    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-zskrf" podStartSLOduration=1.871521318 podStartE2EDuration="6.086512426s" totalImagesPullingTime="4.214991108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:18.085668667 +0000 UTC m=+707.408048833" watchObservedRunningTime="2026-07-18 02:02:18.086512426 +0000 UTC m=+707.408892627"
Jul 18 02:02:18.101872 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:18.101810    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp" podStartSLOduration=2.305195668 podStartE2EDuration="7.101786312s" totalImagesPullingTime="4.796590644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:02:18.099679059 +0000 UTC m=+707.422059251" watchObservedRunningTime="2026-07-18 02:02:18.101786312 +0000 UTC m=+707.424166482"
Jul 18 02:02:29.080261 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:29.080171    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-m72zp"
Jul 18 02:02:46.811045 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:46.811002    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:46.811486 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:46.811232    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-zskrf" podUID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" containerName="authorino" containerID="cri-o://5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de" gracePeriod=30
Jul 18 02:02:47.061857 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.061793    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:47.076576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.076547    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk48l\" (UniqueName: \"kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l\") pod \"b9563bf1-e9b1-49fc-8544-1055d660e1e8\" (UID: \"b9563bf1-e9b1-49fc-8544-1055d660e1e8\") "
Jul 18 02:02:47.079360 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.079325    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l" (OuterVolumeSpecName: "kube-api-access-sk48l") pod "b9563bf1-e9b1-49fc-8544-1055d660e1e8" (UID: "b9563bf1-e9b1-49fc-8544-1055d660e1e8"). InnerVolumeSpecName "kube-api-access-sk48l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:02:47.177611 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.177581    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk48l\" (UniqueName: \"kubernetes.io/projected/b9563bf1-e9b1-49fc-8544-1055d660e1e8-kube-api-access-sk48l\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:02:47.208493 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.208456    2560 generic.go:356] "Generic (PLEG): container finished" podID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" containerID="5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de" exitCode=0
Jul 18 02:02:47.208716 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.208500    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-zskrf"
Jul 18 02:02:47.208716 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.208547    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zskrf" event={"ID":"b9563bf1-e9b1-49fc-8544-1055d660e1e8","Type":"ContainerDied","Data":"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"}
Jul 18 02:02:47.208716 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.208602    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-zskrf" event={"ID":"b9563bf1-e9b1-49fc-8544-1055d660e1e8","Type":"ContainerDied","Data":"47be73c88149a163a4771ec17f415b80aeb3f8f432c5f20641aa007c763c6be7"}
Jul 18 02:02:47.208716 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.208624    2560 scope.go:117] "RemoveContainer" containerID="5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"
Jul 18 02:02:47.217379 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.217361    2560 scope.go:117] "RemoveContainer" containerID="5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"
Jul 18 02:02:47.217664 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:02:47.217644    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de\": container with ID starting with 5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de not found: ID does not exist" containerID="5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"
Jul 18 02:02:47.217741 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.217672    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de"} err="failed to get container status \"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de\": rpc error: code = NotFound desc = could not find container \"5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de\": container with ID starting with 5e66b5e36e8a2e936309ff3ed07561375e6802ac7f31c15ae583e0581d08c0de not found: ID does not exist"
Jul 18 02:02:47.225498 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.225472    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:47.229258 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:47.229229    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-zskrf"]
Jul 18 02:02:49.116009 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:02:49.115975    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" path="/var/lib/kubelet/pods/b9563bf1-e9b1-49fc-8544-1055d660e1e8/volumes"
Jul 18 02:03:28.743426 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:03:28.743384    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:03:46.406382 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:03:46.406343    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:03:50.693504 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:03:50.693467    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:04:01.590056 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.590018    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"]
Jul 18 02:04:01.590648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.590630    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" containerName="authorino"
Jul 18 02:04:01.590746 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.590651    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" containerName="authorino"
Jul 18 02:04:01.590803 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.590745    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9563bf1-e9b1-49fc-8544-1055d660e1e8" containerName="authorino"
Jul 18 02:04:01.593836 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.593814    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.596205 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.596181    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 18 02:04:01.596318 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.596240    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 18 02:04:01.597226 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.597207    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 18 02:04:01.597322 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.597239    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7rth8\""
Jul 18 02:04:01.603586 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.603560    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"]
Jul 18 02:04:01.779025 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.778980    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.779025 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.779026    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cltlb\" (UniqueName: \"kubernetes.io/projected/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-kube-api-access-cltlb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.779253 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.779045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.779253 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.779101    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.779253 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.779171    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.779253 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.779233    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880341 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880310    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880361    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880385    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cltlb\" (UniqueName: \"kubernetes.io/projected/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-kube-api-access-cltlb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880422    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880473    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880893 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880870    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880893 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880881    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.880982 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.880924    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.882817 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.882795    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.883049 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.883032    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.887462 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.887441    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cltlb\" (UniqueName: \"kubernetes.io/projected/297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0-kube-api-access-cltlb\") pod \"e2e-trlp-test-simulated-kserve-598466548d-cvn4q\" (UID: \"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:01.904381 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:01.904349    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:02.044309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:02.044276    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"]
Jul 18 02:04:02.564683 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:02.564642    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q" event={"ID":"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0","Type":"ContainerStarted","Data":"f8445205dc28bc79482fe30a854bdef4a93c2bd8b696ba604b76fcfe27d30724"}
Jul 18 02:04:02.780747 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:02.780704    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:04:03.571658 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:03.571622    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q" event={"ID":"297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0","Type":"ContainerStarted","Data":"bb767b86308d17f20bddf9cc7cdd301aa9c69b188f28204c361f098a3dfdb255"}
Jul 18 02:04:03.591446 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:03.591390    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q" podStartSLOduration=1.570121466 podStartE2EDuration="2.591373934s" totalImagesPullingTime="1.021252468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:03.588953098 +0000 UTC m=+812.911333265" watchObservedRunningTime="2026-07-18 02:04:03.591373934 +0000 UTC m=+812.913754097"
Jul 18 02:04:10.883639 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.883600    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"]
Jul 18 02:04:10.887840 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.887813    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.890078 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.890054    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 18 02:04:10.897225 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.897200    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"]
Jul 18 02:04:10.962874 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.962839    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.962874 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.962879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqmzl\" (UniqueName: \"kubernetes.io/projected/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-kube-api-access-jqmzl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.963097 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.962901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.963097 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.962948    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.963097 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.963028    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:10.963097 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:10.963043    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064021 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.063971    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064078    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064098    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064125    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064145    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqmzl\" (UniqueName: \"kubernetes.io/projected/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-kube-api-access-jqmzl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064309 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064175    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064628 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064502    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064628 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064581    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.064715 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.064642    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.066602 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.066579    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.066875 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.066856    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.072504 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.072481    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqmzl\" (UniqueName: \"kubernetes.io/projected/0a9e7db5-b6cf-4905-8049-cb0ba3d21c15-kube-api-access-jqmzl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99\" (UID: \"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.201774 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.201669    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:11.345840 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.345813    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"]
Jul 18 02:04:11.348148 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:04:11.348121    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a9e7db5_b6cf_4905_8049_cb0ba3d21c15.slice/crio-9b73ceb607e64266cb83c24733a54c5e8ebfaed7d5ea38aa5e5ea6177cf4848c WatchSource:0}: Error finding container 9b73ceb607e64266cb83c24733a54c5e8ebfaed7d5ea38aa5e5ea6177cf4848c: Status 404 returned error can't find the container with id 9b73ceb607e64266cb83c24733a54c5e8ebfaed7d5ea38aa5e5ea6177cf4848c
Jul 18 02:04:11.610637 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.610598    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99" event={"ID":"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15","Type":"ContainerStarted","Data":"9b73ceb607e64266cb83c24733a54c5e8ebfaed7d5ea38aa5e5ea6177cf4848c"}
Jul 18 02:04:11.905187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.905150    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:11.905640 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.905249    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:11.917920 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:11.917890    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:12.484273 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:12.484235    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:04:12.617073 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:12.617033    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99" event={"ID":"0a9e7db5-b6cf-4905-8049-cb0ba3d21c15","Type":"ContainerStarted","Data":"f4b47f0f6ba3a787ecb86ad647c2cb4b9d86f012d17b41461ce8bdf1d42365fd"}
Jul 18 02:04:12.628506 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:12.628476    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-cvn4q"
Jul 18 02:04:12.634935 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:12.634876    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99" podStartSLOduration=2.409761289 podStartE2EDuration="2.634858882s" totalImagesPullingTime="225.097593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:12.634636911 +0000 UTC m=+821.957017080" watchObservedRunningTime="2026-07-18 02:04:12.634858882 +0000 UTC m=+821.957239047"
Jul 18 02:04:19.682702 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:19.682666    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:04:21.202107 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:21.202068    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:21.202564 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:21.202131    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:21.214967 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:21.214935    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:21.672019 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:21.671987    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99"
Jul 18 02:04:27.888198 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:27.888164    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"]
Jul 18 02:04:27.891723 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:27.891707    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:27.893996 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:27.893972    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 18 02:04:27.900956 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:27.900929    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"]
Jul 18 02:04:28.033312 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033275    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.033574 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033326    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b65k\" (UniqueName: \"kubernetes.io/projected/0c8be9d2-f816-4afe-949e-3c7f8d760be8-kube-api-access-8b65k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.033574 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033437    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.033574 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.033574 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033558    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.033748 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.033631    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134588 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134549    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134720 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134608    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134720 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134634    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134720 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134659    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134720 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134714    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.134919 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134757    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b65k\" (UniqueName: \"kubernetes.io/projected/0c8be9d2-f816-4afe-949e-3c7f8d760be8-kube-api-access-8b65k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.135018 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.134991    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.135077 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.135025    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.135122 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.135085    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.137041 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.137023    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c8be9d2-f816-4afe-949e-3c7f8d760be8-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.137294 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.137276    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c8be9d2-f816-4afe-949e-3c7f8d760be8-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.144979 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.144917    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b65k\" (UniqueName: \"kubernetes.io/projected/0c8be9d2-f816-4afe-949e-3c7f8d760be8-kube-api-access-8b65k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z\" (UID: \"0c8be9d2-f816-4afe-949e-3c7f8d760be8\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.202632 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.202592    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:28.340774 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.340746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"]
Jul 18 02:04:28.342824 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:04:28.342793    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c8be9d2_f816_4afe_949e_3c7f8d760be8.slice/crio-7731435b5d97770b41a061a3c7a36d408ccf7d55fd6cb8449dfc4b4f30e948f3 WatchSource:0}: Error finding container 7731435b5d97770b41a061a3c7a36d408ccf7d55fd6cb8449dfc4b4f30e948f3: Status 404 returned error can't find the container with id 7731435b5d97770b41a061a3c7a36d408ccf7d55fd6cb8449dfc4b4f30e948f3
Jul 18 02:04:28.695249 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.695214    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z" event={"ID":"0c8be9d2-f816-4afe-949e-3c7f8d760be8","Type":"ContainerStarted","Data":"896877f7510acb02c6f721b09a4e83e10ff5c2c4a9bb196cf72e5fe39d2788c3"}
Jul 18 02:04:28.695430 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.695259    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z" event={"ID":"0c8be9d2-f816-4afe-949e-3c7f8d760be8","Type":"ContainerStarted","Data":"7731435b5d97770b41a061a3c7a36d408ccf7d55fd6cb8449dfc4b4f30e948f3"}
Jul 18 02:04:28.717221 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:28.712777    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z" podStartSLOduration=1.526825415 podStartE2EDuration="1.712752946s" totalImagesPullingTime="185.927531ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:04:28.710379311 +0000 UTC m=+838.032759479" watchObservedRunningTime="2026-07-18 02:04:28.712752946 +0000 UTC m=+838.035133113"
Jul 18 02:04:30.292823 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:30.292785    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:04:38.203315 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:38.203280    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:38.203822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:38.203339    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:38.216058 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:38.216029    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:04:38.755569 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:04:38.755534    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z"
Jul 18 02:06:17.093124 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:06:17.093081    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:06:26.585348 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:06:26.585307    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:06:37.486292 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:06:37.486250    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:08:20.289538 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:08:20.289488    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:00.092650 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:00.092609    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:10.489510 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:10.489465    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:20.084133 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:20.084090    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:27.400297 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:27.400258    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:37.395570 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:37.395534    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:09:49.887811 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:09:49.887772    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:10:00.784578 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:10:00.784502    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:10:10.604674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:10:10.604639    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:10:35.784393 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:10:35.784351    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:10:47.390129 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:10:47.390073    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:11:10.978778 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:11:10.978744    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:11:21.288710 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:11:21.288673    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:11:35.389001 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:11:35.388912    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:11:46.450978 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:11:46.450947    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:11:56.086373 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:11:56.086338    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:12:05.581226 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:12:05.581183    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:12:14.892024 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:12:14.891987    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:05.090635 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:05.090594    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:16.089699 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:16.089664    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:24.991408 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:24.991308    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:37.492132 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:37.492086    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:49.482865 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:49.482833    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:14:58.383114 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:14:58.383079    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:15:08.396513 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:15:08.396472    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:15:18.185912 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:15:18.185871    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:15:28.983576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:15:28.983539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:15:39.086158 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:15:39.086120    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:15:51.084723 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:15:51.084678    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:16:13.389512 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:16:13.389468    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:16:22.584586 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:16:22.584546    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:16:34.282215 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:16:34.282177    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:16:42.381870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:16:42.381828    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:16:54.409779 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:16:54.409740    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:11.290725 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:11.290680    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:20.587229 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:20.587190    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:29.782865 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:29.782781    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:37.383295 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:37.383250    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:48.298922 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:48.298880    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:17:57.685077 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:17:57.685041    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:18:10.794412 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:18:10.794375    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:18:19.789709 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:18:19.789651    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:18:29.491074 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:18:29.491030    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:18:41.883235 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:18:41.883197    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:18:51.484797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:18:51.484757    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:19:01.393172 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:19:01.393094    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:19:19.392662 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:19:19.392626    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:19:31.483483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:19:31.483443    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:19:42.977893 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:19:42.977857    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:19:56.285251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:19:56.285218    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:20:05.044660 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:20:05.044622    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:20:14.788557 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:20:14.788501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:20:26.184740 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:20:26.184703    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:20:57.497983 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:20:57.497949    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:21:08.501208 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:21:08.501175    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:21:41.616626 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:21:41.616580    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:21:53.013209 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:21:53.013121    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:22:25.574385 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:22:25.574349    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:22:37.934596 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:22:37.934558    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:23:10.933581 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:23:10.933539    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:24:49.544945 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:24:49.544901    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:25:00.524333 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:25:00.524250    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:25:15.992668 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:25:15.992620    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:25:34.096096 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:25:34.096058    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:26:10.500651 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:10.500617    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:26:20.299586 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.299550    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:26:20.303142 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.303123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.307813 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.307791    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2adafcc9\"/\"openshift-service-ca.crt\""
Jul 18 02:26:20.308821 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.308798    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2adafcc9\"/\"default-dockercfg-snvn9\""
Jul 18 02:26:20.308939 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.308838    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2adafcc9\"/\"e2e-lifecycle-model-2adafcc9-kserve-self-signed-certs\""
Jul 18 02:26:20.308939 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.308839    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2adafcc9\"/\"kube-root-ca.crt\""
Jul 18 02:26:20.314539 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.314485    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:26:20.487150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487096    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spr8j\" (UniqueName: \"kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.487150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487153    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.487375 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487214    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.487375 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487267    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.487375 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.487477 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.487380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.587977 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.587885    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spr8j\" (UniqueName: \"kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.587977 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.587929    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.587977 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.587962    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588242 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588003    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588242 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588075    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588242 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588118    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588409 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588376    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588409 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588395    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.588550 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.588509    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.590404 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.590386    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.590602 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.590584    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.595988 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.595965    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spr8j\" (UniqueName: \"kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j\") pod \"e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") " pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.613000 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.612976    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:20.748714 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:20.748685    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:26:20.750937 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:26:20.750901    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8c4ef40_4a9a_4738_8d86_18271d2cf02b.slice/crio-6f6c4dfe460e77dc01bafc2186d74d49a25aea9409c2f1039e1409724fd3db80 WatchSource:0}: Error finding container 6f6c4dfe460e77dc01bafc2186d74d49a25aea9409c2f1039e1409724fd3db80: Status 404 returned error can't find the container with id 6f6c4dfe460e77dc01bafc2186d74d49a25aea9409c2f1039e1409724fd3db80
Jul 18 02:26:21.034721 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:21.034681    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" event={"ID":"a8c4ef40-4a9a-4738-8d86-18271d2cf02b","Type":"ContainerStarted","Data":"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"}
Jul 18 02:26:21.034934 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:21.034728    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" event={"ID":"a8c4ef40-4a9a-4738-8d86-18271d2cf02b","Type":"ContainerStarted","Data":"6f6c4dfe460e77dc01bafc2186d74d49a25aea9409c2f1039e1409724fd3db80"}
Jul 18 02:26:21.053388 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:21.053322    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" podStartSLOduration=1.053305615 podStartE2EDuration="1.053305615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:26:21.051005966 +0000 UTC m=+2150.373386132" watchObservedRunningTime="2026-07-18 02:26:21.053305615 +0000 UTC m=+2150.375685780"
Jul 18 02:26:23.298913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:23.298815    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:26:30.614018 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:30.613976    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:30.616364 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:30.614036    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:30.626770 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:30.626745    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:31.094296 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:31.094260    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:26:35.200784 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:35.200746    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:26:52.389441 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:26:52.389398    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:27:00.608822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:00.608785    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:27:05.805472 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:05.805436    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:27:20.826449 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:20.826402    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" podUID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" containerName="main" containerID="cri-o://ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad" gracePeriod=45
Jul 18 02:27:21.076500 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.076443    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:27:21.181298 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181267    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181313    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181341    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181363    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spr8j\" (UniqueName: \"kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181382    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181483 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181417    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache\") pod \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\" (UID: \"a8c4ef40-4a9a-4738-8d86-18271d2cf02b\") "
Jul 18 02:27:21.181758 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.181645    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:27:21.184391 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.182304    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home" (OuterVolumeSpecName: "home") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:27:21.184391 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.182329    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:27:21.186144 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.184817    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:27:21.186144 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.185089    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j" (OuterVolumeSpecName: "kube-api-access-spr8j") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "kube-api-access-spr8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:27:21.190325 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.190292    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm" (OuterVolumeSpecName: "dshm") pod "a8c4ef40-4a9a-4738-8d86-18271d2cf02b" (UID: "a8c4ef40-4a9a-4738-8d86-18271d2cf02b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:27:21.282995 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.282960    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.282995 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.282991    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.282995 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.282999    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.282995 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.283007    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.283266 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.283016    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spr8j\" (UniqueName: \"kubernetes.io/projected/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-kube-api-access-spr8j\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.283266 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.283025    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a8c4ef40-4a9a-4738-8d86-18271d2cf02b-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:27:21.325605 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.325571    2560 generic.go:356] "Generic (PLEG): container finished" podID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" containerID="ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad" exitCode=0
Jul 18 02:27:21.325737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.325626    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"
Jul 18 02:27:21.325737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.325649    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" event={"ID":"a8c4ef40-4a9a-4738-8d86-18271d2cf02b","Type":"ContainerDied","Data":"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"}
Jul 18 02:27:21.325737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.325691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4" event={"ID":"a8c4ef40-4a9a-4738-8d86-18271d2cf02b","Type":"ContainerDied","Data":"6f6c4dfe460e77dc01bafc2186d74d49a25aea9409c2f1039e1409724fd3db80"}
Jul 18 02:27:21.325737 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.325710    2560 scope.go:117] "RemoveContainer" containerID="ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"
Jul 18 02:27:21.334944 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.334928    2560 scope.go:117] "RemoveContainer" containerID="ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"
Jul 18 02:27:21.335189 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:27:21.335171    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad\": container with ID starting with ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad not found: ID does not exist" containerID="ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"
Jul 18 02:27:21.335251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.335197    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad"} err="failed to get container status \"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad\": rpc error: code = NotFound desc = could not find container \"ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad\": container with ID starting with ffb860b794aae7e4028b0adfc78e63688264c70d6cca70fbde7597ce45e164ad not found: ID does not exist"
Jul 18 02:27:21.347104 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.347083    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:27:21.353089 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:21.353070    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2adafcc9/e2e-lifecycle-model-2adafcc9-kserve-86ddc5b755-ndtr4"]
Jul 18 02:27:23.116777 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:23.116737    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" path="/var/lib/kubelet/pods/a8c4ef40-4a9a-4738-8d86-18271d2cf02b/volumes"
Jul 18 02:27:53.345576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:27:53.345453    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:28:03.410759 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:28:03.410718    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:28:59.702221 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:28:59.702172    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:29:08.223357 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.223324    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:29:08.223795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.223743    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" containerName="main"
Jul 18 02:29:08.223795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.223755    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" containerName="main"
Jul 18 02:29:08.223882 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.223817    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8c4ef40-4a9a-4738-8d86-18271d2cf02b" containerName="main"
Jul 18 02:29:08.226798 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.226778    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.234047 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.234022    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-57b0da\"/\"default-dockercfg-nnjpp\""
Jul 18 02:29:08.234187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.234023    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-57b0da\"/\"test-model-57b0da-kserve-self-signed-certs\""
Jul 18 02:29:08.234187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.234084    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-57b0da\"/\"openshift-service-ca.crt\""
Jul 18 02:29:08.234187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.234125    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-57b0da\"/\"kube-root-ca.crt\""
Jul 18 02:29:08.260238 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.260207    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:29:08.317144 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317106    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.317328 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317154    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.317328 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317177    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xx4h\" (UniqueName: \"kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.317328 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317256    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.317456 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317338    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.317456 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.317362    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418731 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418690    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418749    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418778    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xx4h\" (UniqueName: \"kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418820    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418883    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.418913 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.418909    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.419302 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.419279    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.419302 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.419293    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.419417 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.419280    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.421341 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.421314    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.421502 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.421483    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.429361 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.429331    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xx4h\" (UniqueName: \"kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h\") pod \"test-model-57b0da-kserve-c9cb459bc-f4k4j\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") " pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.533966 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.533869    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:29:08.536573 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.536544    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:08.696541 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.696501    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:29:08.699569 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:29:08.699479    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2dc9077_e494_45d9_979c_a4c281aebbaf.slice/crio-9bcedfb139d6e521a51f4898c4ba18b4467209a35b202cef734f74db7d83ec0c WatchSource:0}: Error finding container 9bcedfb139d6e521a51f4898c4ba18b4467209a35b202cef734f74db7d83ec0c: Status 404 returned error can't find the container with id 9bcedfb139d6e521a51f4898c4ba18b4467209a35b202cef734f74db7d83ec0c
Jul 18 02:29:08.840902 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.840809    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" event={"ID":"e2dc9077-e494-45d9-979c-a4c281aebbaf","Type":"ContainerStarted","Data":"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"}
Jul 18 02:29:08.840902 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.840857    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" event={"ID":"e2dc9077-e494-45d9-979c-a4c281aebbaf","Type":"ContainerStarted","Data":"9bcedfb139d6e521a51f4898c4ba18b4467209a35b202cef734f74db7d83ec0c"}
Jul 18 02:29:08.880610 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:08.880545    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" podStartSLOduration=0.880528377 podStartE2EDuration="880.528377ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:08.879400452 +0000 UTC m=+2318.201780619" watchObservedRunningTime="2026-07-18 02:29:08.880528377 +0000 UTC m=+2318.202908535"
Jul 18 02:29:18.537630 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:18.537584    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:18.538044 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:18.537652    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:18.550565 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:18.550531    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:18.899255 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:18.899221    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:29:19.707609 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:19.707573    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:29:36.806701 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.806614    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:29:36.810053 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.810031    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.813008 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.812984    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f6c952\"/\"default-dockercfg-t44lb\""
Jul 18 02:29:36.813123 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.813036    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f6c952\"/\"openshift-service-ca.crt\""
Jul 18 02:29:36.813964 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.813946    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-f6c952\"/\"kube-root-ca.crt\""
Jul 18 02:29:36.814084 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.813998    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-f6c952\"/\"test-model-f6c952-kserve-self-signed-certs\""
Jul 18 02:29:36.821085 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.821058    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:29:36.894596 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894557    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.894772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894610    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.894772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.894772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.894772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894733    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.894912 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.894791    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x9h9\" (UniqueName: \"kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.995744 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995702    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.995918 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995789    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.995918 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995828    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.995918 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995866    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.995918 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995897    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.996109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.995956    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x9h9\" (UniqueName: \"kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.996166 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.996140    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.996222 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.996165    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.996279 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.996225    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.998308 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.998284    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:36.998489 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:36.998472    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:37.006904 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:37.006871    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x9h9\" (UniqueName: \"kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9\") pod \"test-model-f6c952-kserve-7bd5ccdd79-gvvmw\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") " pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:37.120158 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:37.120072    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:37.284035 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:37.284000    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:29:37.285774 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:29:37.285741    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded0b487d_595f_49a9_8316_e6b10804dab5.slice/crio-e8921e9e666a4c0b5597bfbf096b314507335a2a7d35d3e1e6713e58e4d6874e WatchSource:0}: Error finding container e8921e9e666a4c0b5597bfbf096b314507335a2a7d35d3e1e6713e58e4d6874e: Status 404 returned error can't find the container with id e8921e9e666a4c0b5597bfbf096b314507335a2a7d35d3e1e6713e58e4d6874e
Jul 18 02:29:37.986898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:37.986853    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" event={"ID":"ed0b487d-595f-49a9-8316-e6b10804dab5","Type":"ContainerStarted","Data":"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"}
Jul 18 02:29:37.986898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:37.986900    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" event={"ID":"ed0b487d-595f-49a9-8316-e6b10804dab5","Type":"ContainerStarted","Data":"e8921e9e666a4c0b5597bfbf096b314507335a2a7d35d3e1e6713e58e4d6874e"}
Jul 18 02:29:38.011748 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:38.011696    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" podStartSLOduration=2.011679668 podStartE2EDuration="2.011679668s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:29:38.010091109 +0000 UTC m=+2347.332471276" watchObservedRunningTime="2026-07-18 02:29:38.011679668 +0000 UTC m=+2347.334059842"
Jul 18 02:29:40.901501 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:40.901461    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:29:47.120294 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:47.120258    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:47.120805 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:47.120350    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:47.133370 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:47.133342    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:48.045496 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:48.045469    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:29:55.294586 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:55.294545    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:29:56.661656 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:56.661623    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:29:57.630411 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:29:57.630365    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:30:07.398066 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:07.398024    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:30:11.682436 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:11.682392    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" podUID="e2dc9077-e494-45d9-979c-a4c281aebbaf" containerName="main" containerID="cri-o://2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e" gracePeriod=45
Jul 18 02:30:11.942420 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:11.942347    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:30:12.039928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.039891    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4xx4h\" (UniqueName: \"kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.039965    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040018    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040053    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040090    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040134    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home\") pod \"e2dc9077-e494-45d9-979c-a4c281aebbaf\" (UID: \"e2dc9077-e494-45d9-979c-a4c281aebbaf\") "
Jul 18 02:30:12.040386 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040233    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.040438 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040413    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home" (OuterVolumeSpecName: "home") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.040438 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040420    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.040581 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040498    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.040581 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040548    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.040581 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.040570    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.042465 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.042439    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm" (OuterVolumeSpecName: "dshm") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.042465 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.042447    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h" (OuterVolumeSpecName: "kube-api-access-4xx4h") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "kube-api-access-4xx4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:12.042641 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.042449    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e2dc9077-e494-45d9-979c-a4c281aebbaf" (UID: "e2dc9077-e494-45d9-979c-a4c281aebbaf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:12.142036 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.141996    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2dc9077-e494-45d9-979c-a4c281aebbaf-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.142036 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.142027    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2dc9077-e494-45d9-979c-a4c281aebbaf-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.142036 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.142040    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xx4h\" (UniqueName: \"kubernetes.io/projected/e2dc9077-e494-45d9-979c-a4c281aebbaf-kube-api-access-4xx4h\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.151840 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.151808    2560 generic.go:356] "Generic (PLEG): container finished" podID="e2dc9077-e494-45d9-979c-a4c281aebbaf" containerID="2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e" exitCode=0
Jul 18 02:30:12.151976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.151846    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"
Jul 18 02:30:12.151976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.151875    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" event={"ID":"e2dc9077-e494-45d9-979c-a4c281aebbaf","Type":"ContainerDied","Data":"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"}
Jul 18 02:30:12.151976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.151912    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j" event={"ID":"e2dc9077-e494-45d9-979c-a4c281aebbaf","Type":"ContainerDied","Data":"9bcedfb139d6e521a51f4898c4ba18b4467209a35b202cef734f74db7d83ec0c"}
Jul 18 02:30:12.151976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.151931    2560 scope.go:117] "RemoveContainer" containerID="2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"
Jul 18 02:30:12.161370 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.161347    2560 scope.go:117] "RemoveContainer" containerID="2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"
Jul 18 02:30:12.161676 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:30:12.161651    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e\": container with ID starting with 2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e not found: ID does not exist" containerID="2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"
Jul 18 02:30:12.161795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.161687    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e"} err="failed to get container status \"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e\": rpc error: code = NotFound desc = could not find container \"2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e\": container with ID starting with 2c0fac9bb07aadc2d7423846c692b4c7a8b224728cff46c0a0efdad57ad3f90e not found: ID does not exist"
Jul 18 02:30:12.174328 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.174297    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:30:12.178187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.178159    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-57b0da/test-model-57b0da-kserve-c9cb459bc-f4k4j"]
Jul 18 02:30:12.651136 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.651097    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" podUID="ed0b487d-595f-49a9-8316-e6b10804dab5" containerName="main" containerID="cri-o://6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333" gracePeriod=45
Jul 18 02:30:12.912912 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.912846    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:30:12.949279 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949250    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949421 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949347    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949421 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949399    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949541 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949431    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949541 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949453    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949541 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949488    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9x9h9\" (UniqueName: \"kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9\") pod \"ed0b487d-595f-49a9-8316-e6b10804dab5\" (UID: \"ed0b487d-595f-49a9-8316-e6b10804dab5\") "
Jul 18 02:30:12.949690 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949665    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.949750 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949683    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home" (OuterVolumeSpecName: "home") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.949811 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949788    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:12.949915 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949899    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.949976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949921    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.949976 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.949957    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:12.951878 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.951850    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:30:12.952212 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.952189    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9" (OuterVolumeSpecName: "kube-api-access-9x9h9") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "kube-api-access-9x9h9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:30:12.952212 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:12.952190    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm" (OuterVolumeSpecName: "dshm") pod "ed0b487d-595f-49a9-8316-e6b10804dab5" (UID: "ed0b487d-595f-49a9-8316-e6b10804dab5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:30:13.050598 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.050558    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9x9h9\" (UniqueName: \"kubernetes.io/projected/ed0b487d-595f-49a9-8316-e6b10804dab5-kube-api-access-9x9h9\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:13.050598 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.050596    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed0b487d-595f-49a9-8316-e6b10804dab5-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:13.050598 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.050605    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed0b487d-595f-49a9-8316-e6b10804dab5-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:30:13.116671 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.116635    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2dc9077-e494-45d9-979c-a4c281aebbaf" path="/var/lib/kubelet/pods/e2dc9077-e494-45d9-979c-a4c281aebbaf/volumes"
Jul 18 02:30:13.158294 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.158260    2560 generic.go:356] "Generic (PLEG): container finished" podID="ed0b487d-595f-49a9-8316-e6b10804dab5" containerID="6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333" exitCode=0
Jul 18 02:30:13.158294 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.158297    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"
Jul 18 02:30:13.158505 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.158340    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" event={"ID":"ed0b487d-595f-49a9-8316-e6b10804dab5","Type":"ContainerDied","Data":"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"}
Jul 18 02:30:13.158505 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.158380    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw" event={"ID":"ed0b487d-595f-49a9-8316-e6b10804dab5","Type":"ContainerDied","Data":"e8921e9e666a4c0b5597bfbf096b314507335a2a7d35d3e1e6713e58e4d6874e"}
Jul 18 02:30:13.158505 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.158398    2560 scope.go:117] "RemoveContainer" containerID="6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"
Jul 18 02:30:13.168714 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.168695    2560 scope.go:117] "RemoveContainer" containerID="6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"
Jul 18 02:30:13.168995 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:30:13.168972    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333\": container with ID starting with 6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333 not found: ID does not exist" containerID="6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"
Jul 18 02:30:13.169070 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.169007    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333"} err="failed to get container status \"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333\": rpc error: code = NotFound desc = could not find container \"6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333\": container with ID starting with 6199938d4ad9fe3f1c68aa41facceec619ed429daf5a554ef7f8c3e9be5d5333 not found: ID does not exist"
Jul 18 02:30:13.182291 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.182262    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:30:13.188379 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:13.188358    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-f6c952/test-model-f6c952-kserve-7bd5ccdd79-gvvmw"]
Jul 18 02:30:15.117117 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:15.117081    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed0b487d-595f-49a9-8316-e6b10804dab5" path="/var/lib/kubelet/pods/ed0b487d-595f-49a9-8316-e6b10804dab5/volumes"
Jul 18 02:30:15.403114 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:15.403074    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:30:34.206928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:34.206895    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:30:53.991634 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:30:53.991538    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:31:07.618631 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.618592    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"]
Jul 18 02:31:07.619016 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.618959    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2dc9077-e494-45d9-979c-a4c281aebbaf" containerName="main"
Jul 18 02:31:07.619016 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.618970    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2dc9077-e494-45d9-979c-a4c281aebbaf" containerName="main"
Jul 18 02:31:07.619016 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.618981    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed0b487d-595f-49a9-8316-e6b10804dab5" containerName="main"
Jul 18 02:31:07.619016 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.618987    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed0b487d-595f-49a9-8316-e6b10804dab5" containerName="main"
Jul 18 02:31:07.619150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.619062    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed0b487d-595f-49a9-8316-e6b10804dab5" containerName="main"
Jul 18 02:31:07.619150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.619071    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2dc9077-e494-45d9-979c-a4c281aebbaf" containerName="main"
Jul 18 02:31:07.621873 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.621855    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:07.624973 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.624951    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:31:07.625064 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.624990    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wnbx7\""
Jul 18 02:31:07.633118 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.633090    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"]
Jul 18 02:31:07.656210 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.656164    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w\" (UID: \"e0d25f6a-3245-485c-9d64-4231f9ec89e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:07.756960 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.756921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w\" (UID: \"e0d25f6a-3245-485c-9d64-4231f9ec89e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:07.757576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.757558    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w\" (UID: \"e0d25f6a-3245-485c-9d64-4231f9ec89e4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:07.933478 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:07.933449    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:08.085285 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:08.085108    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"]
Jul 18 02:31:08.087722 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:31:08.087681    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d25f6a_3245_485c_9d64_4231f9ec89e4.slice/crio-b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2 WatchSource:0}: Error finding container b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2: Status 404 returned error can't find the container with id b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2
Jul 18 02:31:08.089671 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:08.089655    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 18 02:31:08.425889 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:08.425855    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w" event={"ID":"e0d25f6a-3245-485c-9d64-4231f9ec89e4","Type":"ContainerStarted","Data":"b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2"}
Jul 18 02:31:10.116507 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:10.116467    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:31:11.441741 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:11.441708    2560 generic.go:356] "Generic (PLEG): container finished" podID="e0d25f6a-3245-485c-9d64-4231f9ec89e4" containerID="c0db0daded92bb6fa2c8135a3933d88b385a3cfa7a7449fb80c743fd1a44c8ae" exitCode=0
Jul 18 02:31:11.442150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:11.441794    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w" event={"ID":"e0d25f6a-3245-485c-9d64-4231f9ec89e4","Type":"ContainerDied","Data":"c0db0daded92bb6fa2c8135a3933d88b385a3cfa7a7449fb80c743fd1a44c8ae"}
Jul 18 02:31:12.580098 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:12.580074    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:12.703591 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:12.703470    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca\") pod \"e0d25f6a-3245-485c-9d64-4231f9ec89e4\" (UID: \"e0d25f6a-3245-485c-9d64-4231f9ec89e4\") "
Jul 18 02:31:12.703833 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:12.703810    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0d25f6a-3245-485c-9d64-4231f9ec89e4" (UID: "e0d25f6a-3245-485c-9d64-4231f9ec89e4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:31:12.804466 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:12.804431    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d25f6a-3245-485c-9d64-4231f9ec89e4-maas-api-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:31:13.453688 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:13.453656    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"
Jul 18 02:31:13.453895 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:13.453691    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w" event={"ID":"e0d25f6a-3245-485c-9d64-4231f9ec89e4","Type":"ContainerDied","Data":"b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2"}
Jul 18 02:31:13.453895 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:13.453739    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b34071a5dc53bd2c668b9efbfc6d9b6849f414b0700111a82ff98e53b2b74ad2"
Jul 18 02:31:26.735452 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:26.735415    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:31:27.731846 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:27.731807    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"]
Jul 18 02:31:27.737941 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:27.737914    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-b8fe6e-r6r2w"]
Jul 18 02:31:29.118176 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:29.118143    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d25f6a-3245-485c-9d64-4231f9ec89e4" path="/var/lib/kubelet/pods/e0d25f6a-3245-485c-9d64-4231f9ec89e4/volumes"
Jul 18 02:31:54.893870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:31:54.893837    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:32:13.096879 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:13.096847    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:32:24.190648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:24.190564    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:32:27.102059 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.102021    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:32:27.102479 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.102420    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d25f6a-3245-485c-9d64-4231f9ec89e4" containerName="revoke-keys"
Jul 18 02:32:27.102479 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.102434    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d25f6a-3245-485c-9d64-4231f9ec89e4" containerName="revoke-keys"
Jul 18 02:32:27.102588 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.102536    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d25f6a-3245-485c-9d64-4231f9ec89e4" containerName="revoke-keys"
Jul 18 02:32:27.105795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.105775    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.108774 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.108744    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"default-dockercfg-8nrhf\""
Jul 18 02:32:27.109681 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.109659    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"openshift-service-ca.crt\""
Jul 18 02:32:27.109792 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.109731    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"kube-root-ca.crt\""
Jul 18 02:32:27.109855 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.109791    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"auth-test-model-4b0cf7-kserve-self-signed-certs\""
Jul 18 02:32:27.130083 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.130051    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:32:27.209286 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209249    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.209286 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209288    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.209514 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209380    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.209514 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.209514 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg4fs\" (UniqueName: \"kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.209648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.209554    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310332 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310302    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310575 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310346    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg4fs\" (UniqueName: \"kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310575 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310403    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310575 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310428    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310575 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310575 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310483    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310892 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310867    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310934 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310897    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.310934 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.310908    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.312894 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.312866    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.313074 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.313059    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.319877 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.319853    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg4fs\" (UniqueName: \"kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs\") pod \"auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") " pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.415740 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.415702    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:27.555464 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.555439    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:32:27.557682 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:32:27.557655    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7de753c3_e2a7_43a7_8d08_e6e0a56aab7f.slice/crio-03bc4928adb964a7e8014e6c72d9ec3472724b070d32cdec730b2e62076f748b WatchSource:0}: Error finding container 03bc4928adb964a7e8014e6c72d9ec3472724b070d32cdec730b2e62076f748b: Status 404 returned error can't find the container with id 03bc4928adb964a7e8014e6c72d9ec3472724b070d32cdec730b2e62076f748b
Jul 18 02:32:27.824567 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.824453    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" event={"ID":"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f","Type":"ContainerStarted","Data":"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"}
Jul 18 02:32:27.824567 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.824502    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" event={"ID":"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f","Type":"ContainerStarted","Data":"03bc4928adb964a7e8014e6c72d9ec3472724b070d32cdec730b2e62076f748b"}
Jul 18 02:32:27.850492 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:27.850437    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" podStartSLOduration=0.850421591 podStartE2EDuration="850.421591ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:32:27.848154661 +0000 UTC m=+2517.170534827" watchObservedRunningTime="2026-07-18 02:32:27.850421591 +0000 UTC m=+2517.172801758"
Jul 18 02:32:31.822676 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:31.822637    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:32:37.416003 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:37.415956    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:37.416475 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:37.416027    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:37.428787 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:37.428754    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:37.895108 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:37.895076    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:32:58.715778 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.715747    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:32:58.719176 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.719157    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.721900 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.721879    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"kube-root-ca.crt\""
Jul 18 02:32:58.722797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.722780    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"default-dockercfg-cj6sq\""
Jul 18 02:32:58.722882 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.722816    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"openshift-service-ca.crt\""
Jul 18 02:32:58.722882 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.722867    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"auth-test-model-124cd9-kserve-self-signed-certs\""
Jul 18 02:32:58.730804 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.730783    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:32:58.813713 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813674    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ksb2\" (UniqueName: \"kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.813898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813724    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.813898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813748    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.813898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813763    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.813898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813780    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.813898 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.813808    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914424 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914388    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914644 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914431    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914644 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914449    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914644 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914468    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914644 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914584    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914864 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914678    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ksb2\" (UniqueName: \"kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914864 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914840    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914969 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914914    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.914969 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.914942    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.916888 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.916868    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.917138 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.917122    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:58.924041 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:58.924020    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ksb2\" (UniqueName: \"kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2\") pod \"auth-test-model-124cd9-kserve-b9d958c99-mgmz7\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") " pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:59.028325 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:59.028233    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:32:59.171879 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:59.171849    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:32:59.174097 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:32:59.174065    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeacc2f1a_4085_45cd_a373_391c3c703f83.slice/crio-4f71cacf47c50930e8e145006bb7a806ebde276c2e08a99af6c7df54574a37f5 WatchSource:0}: Error finding container 4f71cacf47c50930e8e145006bb7a806ebde276c2e08a99af6c7df54574a37f5: Status 404 returned error can't find the container with id 4f71cacf47c50930e8e145006bb7a806ebde276c2e08a99af6c7df54574a37f5
Jul 18 02:32:59.994923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:59.994887    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" event={"ID":"eacc2f1a-4085-45cd-a373-391c3c703f83","Type":"ContainerStarted","Data":"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"}
Jul 18 02:32:59.994923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:32:59.994929    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" event={"ID":"eacc2f1a-4085-45cd-a373-391c3c703f83","Type":"ContainerStarted","Data":"4f71cacf47c50930e8e145006bb7a806ebde276c2e08a99af6c7df54574a37f5"}
Jul 18 02:33:00.014965 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:00.014916    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" podStartSLOduration=2.014898151 podStartE2EDuration="2.014898151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:32:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:33:00.013030031 +0000 UTC m=+2549.335410216" watchObservedRunningTime="2026-07-18 02:33:00.014898151 +0000 UTC m=+2549.337278317"
Jul 18 02:33:02.493047 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:02.493010    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:33:09.029171 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:09.029124    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:33:09.029171 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:09.029180    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:33:09.042179 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:09.042151    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:33:09.053232 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:09.053209    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:33:16.592640 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:16.592604    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:33:34.393382 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:34.393340    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:33:51.102053 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:33:51.102014    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:34:11.099576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:11.099460    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:34:24.796251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:24.796209    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:34:39.297231 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:39.297194    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:34:55.699635 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:55.699600    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:34:56.683000 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:56.682965    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:34:58.790924 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:34:58.790890    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:35:10.720404 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:10.720327    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" podUID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" containerName="main" containerID="cri-o://fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3" gracePeriod=45
Jul 18 02:35:10.983839 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:10.983776    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:35:11.104065 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104011    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg4fs\" (UniqueName: \"kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104095    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104125    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104144    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104251 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104223    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104474 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104289    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache\") pod \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\" (UID: \"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f\") "
Jul 18 02:35:11.104474 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104423    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home" (OuterVolumeSpecName: "home") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:11.104474 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104444    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:11.104670 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104620    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:11.104750 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104730    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.104792 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104755    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.104792 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.104767    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.106581 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.106561    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm" (OuterVolumeSpecName: "dshm") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:11.106681 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.106662    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs" (OuterVolumeSpecName: "kube-api-access-zg4fs") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "kube-api-access-zg4fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:35:11.106723 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.106708    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" (UID: "7de753c3-e2a7-43a7-8d08-e6e0a56aab7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:35:11.205822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.205779    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.205822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.205807    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg4fs\" (UniqueName: \"kubernetes.io/projected/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-kube-api-access-zg4fs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.205822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.205817    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:11.682674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.682632    2560 generic.go:356] "Generic (PLEG): container finished" podID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" containerID="fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3" exitCode=0
Jul 18 02:35:11.682859 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.682741    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"
Jul 18 02:35:11.682859 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.682774    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" event={"ID":"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f","Type":"ContainerDied","Data":"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"}
Jul 18 02:35:11.682859 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.682822    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs" event={"ID":"7de753c3-e2a7-43a7-8d08-e6e0a56aab7f","Type":"ContainerDied","Data":"03bc4928adb964a7e8014e6c72d9ec3472724b070d32cdec730b2e62076f748b"}
Jul 18 02:35:11.682859 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.682844    2560 scope.go:117] "RemoveContainer" containerID="fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"
Jul 18 02:35:11.691884 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.691861    2560 scope.go:117] "RemoveContainer" containerID="fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"
Jul 18 02:35:11.695190 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:35:11.695162    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3\": container with ID starting with fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3 not found: ID does not exist" containerID="fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"
Jul 18 02:35:11.695322 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.695199    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3"} err="failed to get container status \"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3\": rpc error: code = NotFound desc = could not find container \"fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3\": container with ID starting with fd1fa9b513719519d8821fe1adf7d48b8fc562002d327676ba3c8be5ff4ca1d3 not found: ID does not exist"
Jul 18 02:35:11.700204 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.700177    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:35:11.704170 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.704139    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" podUID="eacc2f1a-4085-45cd-a373-391c3c703f83" containerName="main" containerID="cri-o://b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33" gracePeriod=45
Jul 18 02:35:11.704661 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.704641    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/auth-test-model-4b0cf7-kserve-546d8f755f-dnsbs"]
Jul 18 02:35:11.966859 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:11.966835    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:35:12.115221 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115177    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115395 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115262    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115395 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115294    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115395 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115338    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6ksb2\" (UniqueName: \"kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115395 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115384    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115461    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm\") pod \"eacc2f1a-4085-45cd-a373-391c3c703f83\" (UID: \"eacc2f1a-4085-45cd-a373-391c3c703f83\") "
Jul 18 02:35:12.115674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115624    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:12.115674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115639    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home" (OuterVolumeSpecName: "home") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:12.115824 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115796    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.115878 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115832    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.115878 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.115795    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache" (OuterVolumeSpecName: "model-cache") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:12.117745 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.117720    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm" (OuterVolumeSpecName: "dshm") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:35:12.117856 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.117780    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:35:12.117856 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.117818    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2" (OuterVolumeSpecName: "kube-api-access-6ksb2") pod "eacc2f1a-4085-45cd-a373-391c3c703f83" (UID: "eacc2f1a-4085-45cd-a373-391c3c703f83"). InnerVolumeSpecName "kube-api-access-6ksb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:35:12.216553 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.216441    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.216553 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.216473    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eacc2f1a-4085-45cd-a373-391c3c703f83-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.216553 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.216483    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eacc2f1a-4085-45cd-a373-391c3c703f83-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.216553 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.216491    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6ksb2\" (UniqueName: \"kubernetes.io/projected/eacc2f1a-4085-45cd-a373-391c3c703f83-kube-api-access-6ksb2\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:35:12.688794 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.688757    2560 generic.go:356] "Generic (PLEG): container finished" podID="eacc2f1a-4085-45cd-a373-391c3c703f83" containerID="b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33" exitCode=0
Jul 18 02:35:12.688984 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.688800    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"
Jul 18 02:35:12.688984 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.688848    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" event={"ID":"eacc2f1a-4085-45cd-a373-391c3c703f83","Type":"ContainerDied","Data":"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"}
Jul 18 02:35:12.688984 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.688892    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7" event={"ID":"eacc2f1a-4085-45cd-a373-391c3c703f83","Type":"ContainerDied","Data":"4f71cacf47c50930e8e145006bb7a806ebde276c2e08a99af6c7df54574a37f5"}
Jul 18 02:35:12.688984 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.688910    2560 scope.go:117] "RemoveContainer" containerID="b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"
Jul 18 02:35:12.699031 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.699008    2560 scope.go:117] "RemoveContainer" containerID="b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"
Jul 18 02:35:12.699373 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:35:12.699355    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33\": container with ID starting with b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33 not found: ID does not exist" containerID="b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"
Jul 18 02:35:12.699443 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.699383    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33"} err="failed to get container status \"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33\": rpc error: code = NotFound desc = could not find container \"b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33\": container with ID starting with b3645530c0e05b537d6d380be92ff23d3af8535794ee7a8ecd395dfcdf269b33 not found: ID does not exist"
Jul 18 02:35:12.710424 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.710390    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:35:12.715862 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:12.715832    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/auth-test-model-124cd9-kserve-b9d958c99-mgmz7"]
Jul 18 02:35:13.116158 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:13.116062    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" path="/var/lib/kubelet/pods/7de753c3-e2a7-43a7-8d08-e6e0a56aab7f/volumes"
Jul 18 02:35:13.116674 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:13.116652    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eacc2f1a-4085-45cd-a373-391c3c703f83" path="/var/lib/kubelet/pods/eacc2f1a-4085-45cd-a373-391c3c703f83/volumes"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.110734    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111599    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eacc2f1a-4085-45cd-a373-391c3c703f83" containerName="main"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111621    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eacc2f1a-4085-45cd-a373-391c3c703f83" containerName="main"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111672    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" containerName="main"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111681    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" containerName="main"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111848    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="7de753c3-e2a7-43a7-8d08-e6e0a56aab7f" containerName="main"
Jul 18 02:35:18.113757 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.111873    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eacc2f1a-4085-45cd-a373-391c3c703f83" containerName="main"
Jul 18 02:35:18.117176 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.117149    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.120053 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.120029    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"default-dockercfg-8nrhf\""
Jul 18 02:35:18.120187 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.120053    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"openshift-service-ca.crt\""
Jul 18 02:35:18.120964 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.120944    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"sub-test-model-4b0cf7-kserve-self-signed-certs\""
Jul 18 02:35:18.121066 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.120982    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"kube-root-ca.crt\""
Jul 18 02:35:18.127501 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.127478    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:35:18.276398 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276358    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.276626 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7676\" (UniqueName: \"kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.276626 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276462    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.276626 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276559    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.276626 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.276772 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.276647    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377799 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377715    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377799 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377762    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377989 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377815    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377989 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377862    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7676\" (UniqueName: \"kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377989 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.377989 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.377936    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.378206 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.378186    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.378241 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.378217    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.378275 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.378240    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.380380 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.380350    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.380632 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.380616    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.387390 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.387366    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7676\" (UniqueName: \"kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676\") pod \"sub-test-model-4b0cf7-kserve-c554d858c-5cwfj\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") " pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.426462 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.426417    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:18.568935 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.568904    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:35:18.570745 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:35:18.570718    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22ed39db_f28e_4eca_aa87_6de7ba07cba6.slice/crio-dc52d895d5f86a5c62bc45b639a266f0adc180022a5e02abb3343ac03702d9fb WatchSource:0}: Error finding container dc52d895d5f86a5c62bc45b639a266f0adc180022a5e02abb3343ac03702d9fb: Status 404 returned error can't find the container with id dc52d895d5f86a5c62bc45b639a266f0adc180022a5e02abb3343ac03702d9fb
Jul 18 02:35:18.720189 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.720155    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" event={"ID":"22ed39db-f28e-4eca-aa87-6de7ba07cba6","Type":"ContainerStarted","Data":"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"}
Jul 18 02:35:18.720407 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.720200    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" event={"ID":"22ed39db-f28e-4eca-aa87-6de7ba07cba6","Type":"ContainerStarted","Data":"dc52d895d5f86a5c62bc45b639a266f0adc180022a5e02abb3343ac03702d9fb"}
Jul 18 02:35:18.742999 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:18.742941    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" podStartSLOduration=0.742924007 podStartE2EDuration="742.924007ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:35:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:35:18.742578315 +0000 UTC m=+2688.064958494" watchObservedRunningTime="2026-07-18 02:35:18.742924007 +0000 UTC m=+2688.065304172"
Jul 18 02:35:20.499132 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:20.499094    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:35:28.426972 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:28.426883    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:28.427448 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:28.426965    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:28.439375 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:28.439354    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:28.785387 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:28.785298    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:35:49.251616 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.251581    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:35:49.255290 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.255270    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.259629 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.259602    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"default-dockercfg-cj6sq\""
Jul 18 02:35:49.259629 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.259628    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"openshift-service-ca.crt\""
Jul 18 02:35:49.259824 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.259603    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"sub-test-model-124cd9-kserve-self-signed-certs\""
Jul 18 02:35:49.259824 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.259603    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"kube-root-ca.crt\""
Jul 18 02:35:49.268267 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.268239    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:35:49.371791 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.371747    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.371992 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.371807    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.371992 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.371867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.371992 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.371943    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.372130 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.371998    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.372130 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.372077    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh6dw\" (UniqueName: \"kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473361 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473317    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh6dw\" (UniqueName: \"kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473372    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473410    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473444    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473481    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473509    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473939 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473915    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.473939 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473935    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.474106 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.473994    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.475827 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.475808    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.476176 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.476154    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.481378 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.481354    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh6dw\" (UniqueName: \"kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw\") pod \"sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") " pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.565139 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.565044    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:49.712240 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.712211    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:35:49.714556 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:35:49.714498    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e3fb2d2_4371_4001_b003_621e70bec22a.slice/crio-721233a32246a21d92604822a7e517fde9ada8631c43ef0212c1f21f00c01e59 WatchSource:0}: Error finding container 721233a32246a21d92604822a7e517fde9ada8631c43ef0212c1f21f00c01e59: Status 404 returned error can't find the container with id 721233a32246a21d92604822a7e517fde9ada8631c43ef0212c1f21f00c01e59
Jul 18 02:35:49.884648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.884616    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" event={"ID":"8e3fb2d2-4371-4001-b003-621e70bec22a","Type":"ContainerStarted","Data":"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"}
Jul 18 02:35:49.884801 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.884659    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" event={"ID":"8e3fb2d2-4371-4001-b003-621e70bec22a","Type":"ContainerStarted","Data":"721233a32246a21d92604822a7e517fde9ada8631c43ef0212c1f21f00c01e59"}
Jul 18 02:35:49.902442 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:49.902393    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" podStartSLOduration=0.90237491 podStartE2EDuration="902.37491ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:35:49.901107682 +0000 UTC m=+2719.223487848" watchObservedRunningTime="2026-07-18 02:35:49.90237491 +0000 UTC m=+2719.224755110"
Jul 18 02:35:52.494092 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:52.494051    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:35:59.566140 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:59.566099    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:59.566682 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:59.566154    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:59.579077 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:59.579045    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:35:59.949747 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:35:59.949710    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:36:08.903449 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:08.903413    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:36:23.596865 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:23.596830    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:36:24.501787 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:24.501744    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:36:29.692432 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:29.692397    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:36:38.101031 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.100997    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:36:38.104756 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.104738    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.107357 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.107334    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4b0cf7\"/\"rate-test-model-4b0cf7-kserve-self-signed-certs\""
Jul 18 02:36:38.114689 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.114669    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:36:38.258692 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258648    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.258870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258732    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5hsx\" (UniqueName: \"kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.258870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258758    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.258870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258777    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.258870 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258836    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.259004 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.258875    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359463    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359576 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359556    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359595    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359642    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5hsx\" (UniqueName: \"kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359679    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359797 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359728    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.359994 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.359938    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.360052 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.360033    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.360101 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.360079    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.362017 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.361987    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.362234 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.362215    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.369288 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.369266    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5hsx\" (UniqueName: \"kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx\") pod \"rate-test-model-4b0cf7-kserve-7767c55887-rtjj8\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") " pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.415225 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.415192    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:38.551129 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.551101    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:36:38.552647 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:36:38.552623    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb1b6035_7ecf_475b_9033_f8b33287d102.slice/crio-a3e875bd15e55959b8b6e1180593a2eba417ed9ec2736250b20168fc0a533d3e WatchSource:0}: Error finding container a3e875bd15e55959b8b6e1180593a2eba417ed9ec2736250b20168fc0a533d3e: Status 404 returned error can't find the container with id a3e875bd15e55959b8b6e1180593a2eba417ed9ec2736250b20168fc0a533d3e
Jul 18 02:36:38.617137 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.617102    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerName="main" containerID="cri-o://5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3" gracePeriod=45
Jul 18 02:36:38.775030 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.774990    2560 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.45:8000/ready\": dial tcp 10.132.0.45:8000: connect: connection refused"
Jul 18 02:36:38.850354 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.850325    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:36:38.863557 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863442    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.863557 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863559    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7676\" (UniqueName: \"kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.863775 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863646    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.863775 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863675    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.863775 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863717    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.863775 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863749    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home" (OuterVolumeSpecName: "home") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:38.863775 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863773    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs\") pod \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\" (UID: \"22ed39db-f28e-4eca-aa87-6de7ba07cba6\") "
Jul 18 02:36:38.864007 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863923    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache" (OuterVolumeSpecName: "model-cache") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:38.864007 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.863938    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:38.864173 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.864150    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:38.864173 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.864171    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:38.864275 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.864180    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:38.865811 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.865789    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676" (OuterVolumeSpecName: "kube-api-access-c7676") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "kube-api-access-c7676". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:36:38.865896 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.865858    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:36:38.865896 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.865873    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm" (OuterVolumeSpecName: "dshm") pod "22ed39db-f28e-4eca-aa87-6de7ba07cba6" (UID: "22ed39db-f28e-4eca-aa87-6de7ba07cba6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:38.965629 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.965589    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/22ed39db-f28e-4eca-aa87-6de7ba07cba6-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:38.965629 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.965630    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7676\" (UniqueName: \"kubernetes.io/projected/22ed39db-f28e-4eca-aa87-6de7ba07cba6-kube-api-access-c7676\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:38.965629 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:38.965641    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/22ed39db-f28e-4eca-aa87-6de7ba07cba6-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.129013 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.128977    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" event={"ID":"fb1b6035-7ecf-475b-9033-f8b33287d102","Type":"ContainerStarted","Data":"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"}
Jul 18 02:36:39.129375 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.129027    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" event={"ID":"fb1b6035-7ecf-475b-9033-f8b33287d102","Type":"ContainerStarted","Data":"a3e875bd15e55959b8b6e1180593a2eba417ed9ec2736250b20168fc0a533d3e"}
Jul 18 02:36:39.130315 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.130293    2560 generic.go:356] "Generic (PLEG): container finished" podID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerID="5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3" exitCode=0
Jul 18 02:36:39.130388 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.130334    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"
Jul 18 02:36:39.130388 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.130365    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" event={"ID":"22ed39db-f28e-4eca-aa87-6de7ba07cba6","Type":"ContainerDied","Data":"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"}
Jul 18 02:36:39.130461 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.130403    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj" event={"ID":"22ed39db-f28e-4eca-aa87-6de7ba07cba6","Type":"ContainerDied","Data":"dc52d895d5f86a5c62bc45b639a266f0adc180022a5e02abb3343ac03702d9fb"}
Jul 18 02:36:39.130461 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.130420    2560 scope.go:117] "RemoveContainer" containerID="5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"
Jul 18 02:36:39.147335 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.147313    2560 scope.go:117] "RemoveContainer" containerID="5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"
Jul 18 02:36:39.147704 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:36:39.147670    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3\": container with ID starting with 5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3 not found: ID does not exist" containerID="5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"
Jul 18 02:36:39.147886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.147717    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3"} err="failed to get container status \"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3\": rpc error: code = NotFound desc = could not find container \"5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3\": container with ID starting with 5ea1ee79b2a290e4461ac12c2fe289bf44ca78fd63017cf7ffba37a5be0362b3 not found: ID does not exist"
Jul 18 02:36:39.149471 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.149428    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" podStartSLOduration=1.149413351 podStartE2EDuration="1.149413351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:36:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:36:39.147870147 +0000 UTC m=+2768.470250311" watchObservedRunningTime="2026-07-18 02:36:39.149413351 +0000 UTC m=+2768.471793518"
Jul 18 02:36:39.160702 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.160680    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:36:39.166274 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.166250    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/sub-test-model-4b0cf7-kserve-c554d858c-5cwfj"]
Jul 18 02:36:39.523353 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.523317    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" podUID="8e3fb2d2-4371-4001-b003-621e70bec22a" containerName="main" containerID="cri-o://4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d" gracePeriod=45
Jul 18 02:36:39.784028 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.783964    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:36:39.874383 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874352    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874383 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874387    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874620 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874432    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rh6dw\" (UniqueName: \"kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874620 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874494    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874620 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874573    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874620 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874619    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm\") pod \"8e3fb2d2-4371-4001-b003-621e70bec22a\" (UID: \"8e3fb2d2-4371-4001-b003-621e70bec22a\") "
Jul 18 02:36:39.874830 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874673    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:39.874830 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874729    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home" (OuterVolumeSpecName: "home") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:39.874982 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874841    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache" (OuterVolumeSpecName: "model-cache") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:39.875038 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.874991    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.875038 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.875011    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.875038 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.875023    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.876755 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.876735    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm" (OuterVolumeSpecName: "dshm") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:36:39.876840 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.876754    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw" (OuterVolumeSpecName: "kube-api-access-rh6dw") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "kube-api-access-rh6dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:36:39.876878 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.876835    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8e3fb2d2-4371-4001-b003-621e70bec22a" (UID: "8e3fb2d2-4371-4001-b003-621e70bec22a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:36:39.975999 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.975956    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e3fb2d2-4371-4001-b003-621e70bec22a-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.975999 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.975991    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8e3fb2d2-4371-4001-b003-621e70bec22a-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:39.975999 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:39.976001    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rh6dw\" (UniqueName: \"kubernetes.io/projected/8e3fb2d2-4371-4001-b003-621e70bec22a-kube-api-access-rh6dw\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:36:40.136481 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.136446    2560 generic.go:356] "Generic (PLEG): container finished" podID="8e3fb2d2-4371-4001-b003-621e70bec22a" containerID="4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d" exitCode=0
Jul 18 02:36:40.136980 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.136495    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"
Jul 18 02:36:40.136980 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.136595    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" event={"ID":"8e3fb2d2-4371-4001-b003-621e70bec22a","Type":"ContainerDied","Data":"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"}
Jul 18 02:36:40.136980 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.136647    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9" event={"ID":"8e3fb2d2-4371-4001-b003-621e70bec22a","Type":"ContainerDied","Data":"721233a32246a21d92604822a7e517fde9ada8631c43ef0212c1f21f00c01e59"}
Jul 18 02:36:40.136980 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.136664    2560 scope.go:117] "RemoveContainer" containerID="4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"
Jul 18 02:36:40.146029 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.146014    2560 scope.go:117] "RemoveContainer" containerID="4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"
Jul 18 02:36:40.146312 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:36:40.146287    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d\": container with ID starting with 4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d not found: ID does not exist" containerID="4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"
Jul 18 02:36:40.146412 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.146319    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d"} err="failed to get container status \"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d\": rpc error: code = NotFound desc = could not find container \"4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d\": container with ID starting with 4ad4066473b46016698b2c9411d4d0b3c6b9c9fff5b0b6d5b7d1c1af1bac8f0d not found: ID does not exist"
Jul 18 02:36:40.157972 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.157944    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:36:40.161152 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:40.161124    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/sub-test-model-124cd9-kserve-7bdd4bf7bf-9kwt9"]
Jul 18 02:36:41.115915 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:41.115879    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" path="/var/lib/kubelet/pods/22ed39db-f28e-4eca-aa87-6de7ba07cba6/volumes"
Jul 18 02:36:41.116265 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:41.116252    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e3fb2d2-4371-4001-b003-621e70bec22a" path="/var/lib/kubelet/pods/8e3fb2d2-4371-4001-b003-621e70bec22a/volumes"
Jul 18 02:36:42.192993 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:42.192958    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:36:48.415995 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:48.415942    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:48.416513 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:48.416020    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:48.428367 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:48.428340    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:36:49.192718 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:36:49.192685    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:37:05.308514 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308479    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308869    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerName="main"
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308881    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerName="main"
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308897    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e3fb2d2-4371-4001-b003-621e70bec22a" containerName="main"
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308903    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e3fb2d2-4371-4001-b003-621e70bec22a" containerName="main"
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308964    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e3fb2d2-4371-4001-b003-621e70bec22a" containerName="main"
Jul 18 02:37:05.310886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.308975    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="22ed39db-f28e-4eca-aa87-6de7ba07cba6" containerName="main"
Jul 18 02:37:05.311776 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.311761    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.315053 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.315031    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"rate-test-model-124cd9-kserve-self-signed-certs\""
Jul 18 02:37:05.315926 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.315907    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"openshift-service-ca.crt\""
Jul 18 02:37:05.316064 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.316044    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"default-dockercfg-cj6sq\""
Jul 18 02:37:05.316136 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.316047    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-124cd9\"/\"kube-root-ca.crt\""
Jul 18 02:37:05.330050 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.330009    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:37:05.427749 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.427704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.427955 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.427760    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.427955 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.427785    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.427955 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.427879    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhqnd\" (UniqueName: \"kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.428130 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.427967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.428130 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.428018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.528911 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.528874    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.528921    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.528945    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.528970    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhqnd\" (UniqueName: \"kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.529006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529126 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.529028    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529403 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.529357    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529403 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.529361    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.529484 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.529403    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.531579 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.531549    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.531899 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.531879    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.538152 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.538123    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhqnd\" (UniqueName: \"kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd\") pod \"rate-test-model-124cd9-kserve-6868b95ccf-qskh7\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") " pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.627997 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.627907    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:05.765816 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:05.765783    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:37:05.769599 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:37:05.769554    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43871f23_ac25_408d_9815_0b78de1069d0.slice/crio-7bbca077caa27c5beccd9ebb4b8a2c5ac5fc02112afc56c6d497f7cfaf00f4c8 WatchSource:0}: Error finding container 7bbca077caa27c5beccd9ebb4b8a2c5ac5fc02112afc56c6d497f7cfaf00f4c8: Status 404 returned error can't find the container with id 7bbca077caa27c5beccd9ebb4b8a2c5ac5fc02112afc56c6d497f7cfaf00f4c8
Jul 18 02:37:06.266433 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:06.266396    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" event={"ID":"43871f23-ac25-408d-9815-0b78de1069d0","Type":"ContainerStarted","Data":"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"}
Jul 18 02:37:06.266637 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:06.266441    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" event={"ID":"43871f23-ac25-408d-9815-0b78de1069d0","Type":"ContainerStarted","Data":"7bbca077caa27c5beccd9ebb4b8a2c5ac5fc02112afc56c6d497f7cfaf00f4c8"}
Jul 18 02:37:06.284441 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:06.284380    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" podStartSLOduration=1.284357577 podStartE2EDuration="1.284357577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-18 02:37:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:37:06.282311965 +0000 UTC m=+2795.604692130" watchObservedRunningTime="2026-07-18 02:37:06.284357577 +0000 UTC m=+2795.606737743"
Jul 18 02:37:09.194623 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:09.194588    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:37:15.628692 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:15.628654    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:15.629092 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:15.628732    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:15.641174 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:15.641147    2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:16.327191 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:16.327161    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:37:23.095955 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:23.095915    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:37:31.354841 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:31.354807    2560 scope.go:117] "RemoveContainer" containerID="c0db0daded92bb6fa2c8135a3933d88b385a3cfa7a7449fb80c743fd1a44c8ae"
Jul 18 02:37:37.992803 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:37.992770    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:37:43.848137 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:43.848101    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:37:44.681070 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:44.681035    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:37:55.692379 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:55.692339    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:37:58.868822 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:58.868761    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" podUID="fb1b6035-7ecf-475b-9033-f8b33287d102" containerName="main" containerID="cri-o://ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206" gracePeriod=45
Jul 18 02:37:59.134628 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.134603    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:37:59.153727 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153696    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.153928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153771    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.153928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153811    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.153928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153870    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5hsx\" (UniqueName: \"kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.153928 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153923    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.154166 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.153966    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home\") pod \"fb1b6035-7ecf-475b-9033-f8b33287d102\" (UID: \"fb1b6035-7ecf-475b-9033-f8b33287d102\") "
Jul 18 02:37:59.154166 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.154013    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:37:59.154298 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.154283    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.156039 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.154488    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home" (OuterVolumeSpecName: "home") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:37:59.156039 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.156002    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache" (OuterVolumeSpecName: "model-cache") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:37:59.156902 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.156864    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:37:59.157235 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.157212    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm" (OuterVolumeSpecName: "dshm") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:37:59.157769 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.157742    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx" (OuterVolumeSpecName: "kube-api-access-z5hsx") pod "fb1b6035-7ecf-475b-9033-f8b33287d102" (UID: "fb1b6035-7ecf-475b-9033-f8b33287d102"). InnerVolumeSpecName "kube-api-access-z5hsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:37:59.256554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.255705    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.256554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.255750    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb1b6035-7ecf-475b-9033-f8b33287d102-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.256554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.255808    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5hsx\" (UniqueName: \"kubernetes.io/projected/fb1b6035-7ecf-475b-9033-f8b33287d102-kube-api-access-z5hsx\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.256554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.255843    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.256554 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.255858    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fb1b6035-7ecf-475b-9033-f8b33287d102-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:37:59.532685 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.532588    2560 generic.go:356] "Generic (PLEG): container finished" podID="fb1b6035-7ecf-475b-9033-f8b33287d102" containerID="ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206" exitCode=0
Jul 18 02:37:59.532685 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.532656    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" event={"ID":"fb1b6035-7ecf-475b-9033-f8b33287d102","Type":"ContainerDied","Data":"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"}
Jul 18 02:37:59.532685 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.532683    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8" event={"ID":"fb1b6035-7ecf-475b-9033-f8b33287d102","Type":"ContainerDied","Data":"a3e875bd15e55959b8b6e1180593a2eba417ed9ec2736250b20168fc0a533d3e"}
Jul 18 02:37:59.532923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.532700    2560 scope.go:117] "RemoveContainer" containerID="ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"
Jul 18 02:37:59.532923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.532655    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"
Jul 18 02:37:59.541886 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.541869    2560 scope.go:117] "RemoveContainer" containerID="ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"
Jul 18 02:37:59.542151 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:37:59.542132    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206\": container with ID starting with ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206 not found: ID does not exist" containerID="ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"
Jul 18 02:37:59.542189 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.542161    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206"} err="failed to get container status \"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206\": rpc error: code = NotFound desc = could not find container \"ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206\": container with ID starting with ab16b3861f1ace8c2737ca37b6aac2e6245bc8b079348c50067d03e91e3c8206 not found: ID does not exist"
Jul 18 02:37:59.573880 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.573833    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:37:59.576222 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.576194    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4b0cf7/rate-test-model-4b0cf7-kserve-7767c55887-rtjj8"]
Jul 18 02:37:59.701980 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.701930    2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" podUID="43871f23-ac25-408d-9815-0b78de1069d0" containerName="main" containerID="cri-o://9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50" gracePeriod=45
Jul 18 02:37:59.959123 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:37:59.959095    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:38:00.063551 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063505    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhqnd\" (UniqueName: \"kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.063551 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063558    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.063810 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063580    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.063810 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063618    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.063810 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063663    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.063810 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063732    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir\") pod \"43871f23-ac25-408d-9815-0b78de1069d0\" (UID: \"43871f23-ac25-408d-9815-0b78de1069d0\") "
Jul 18 02:38:00.064013 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063851    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:38:00.064013 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.063924    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home" (OuterVolumeSpecName: "home") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:38:00.064111 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.064053    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:38:00.064111 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.064095    2560 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-model-cache\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.064194 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.064114    2560 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-home\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.066060 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.066005    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 18 02:38:00.066060 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.066007    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm" (OuterVolumeSpecName: "dshm") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 18 02:38:00.066175 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.066094    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd" (OuterVolumeSpecName: "kube-api-access-dhqnd") pod "43871f23-ac25-408d-9815-0b78de1069d0" (UID: "43871f23-ac25-408d-9815-0b78de1069d0"). InnerVolumeSpecName "kube-api-access-dhqnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 18 02:38:00.165354 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.165316    2560 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-dshm\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.165354 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.165345    2560 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/43871f23-ac25-408d-9815-0b78de1069d0-tls-certs\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.165354 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.165355    2560 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/43871f23-ac25-408d-9815-0b78de1069d0-tmp-dir\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.165354 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.165363    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhqnd\" (UniqueName: \"kubernetes.io/projected/43871f23-ac25-408d-9815-0b78de1069d0-kube-api-access-dhqnd\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:00.539167 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.539129    2560 generic.go:356] "Generic (PLEG): container finished" podID="43871f23-ac25-408d-9815-0b78de1069d0" containerID="9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50" exitCode=0
Jul 18 02:38:00.539347 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.539175    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"
Jul 18 02:38:00.539347 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.539206    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" event={"ID":"43871f23-ac25-408d-9815-0b78de1069d0","Type":"ContainerDied","Data":"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"}
Jul 18 02:38:00.539347 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.539242    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7" event={"ID":"43871f23-ac25-408d-9815-0b78de1069d0","Type":"ContainerDied","Data":"7bbca077caa27c5beccd9ebb4b8a2c5ac5fc02112afc56c6d497f7cfaf00f4c8"}
Jul 18 02:38:00.539347 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.539264    2560 scope.go:117] "RemoveContainer" containerID="9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"
Jul 18 02:38:00.548328 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.548306    2560 scope.go:117] "RemoveContainer" containerID="9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"
Jul 18 02:38:00.548644 ip-10-0-129-141 kubenswrapper[2560]: E0718 02:38:00.548624    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50\": container with ID starting with 9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50 not found: ID does not exist" containerID="9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"
Jul 18 02:38:00.548708 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.548653    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50"} err="failed to get container status \"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50\": rpc error: code = NotFound desc = could not find container \"9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50\": container with ID starting with 9394cf2cd311fc4ce77aa757504285df29a61abc7a40100718c3b06da0330a50 not found: ID does not exist"
Jul 18 02:38:00.560199 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.560165    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:38:00.566386 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:00.566358    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-124cd9/rate-test-model-124cd9-kserve-6868b95ccf-qskh7"]
Jul 18 02:38:01.117199 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:01.117164    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43871f23-ac25-408d-9815-0b78de1069d0" path="/var/lib/kubelet/pods/43871f23-ac25-408d-9815-0b78de1069d0/volumes"
Jul 18 02:38:01.117592 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:01.117568    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb1b6035-7ecf-475b-9033-f8b33287d102" path="/var/lib/kubelet/pods/fb1b6035-7ecf-475b-9033-f8b33287d102/volumes"
Jul 18 02:38:04.133190 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133150    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"]
Jul 18 02:38:04.133636 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133596    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43871f23-ac25-408d-9815-0b78de1069d0" containerName="main"
Jul 18 02:38:04.133636 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133614    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="43871f23-ac25-408d-9815-0b78de1069d0" containerName="main"
Jul 18 02:38:04.133636 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133635    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb1b6035-7ecf-475b-9033-f8b33287d102" containerName="main"
Jul 18 02:38:04.133765 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133643    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb1b6035-7ecf-475b-9033-f8b33287d102" containerName="main"
Jul 18 02:38:04.133765 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133737    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb1b6035-7ecf-475b-9033-f8b33287d102" containerName="main"
Jul 18 02:38:04.133765 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.133754    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="43871f23-ac25-408d-9815-0b78de1069d0" containerName="main"
Jul 18 02:38:04.138222 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.138204    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:04.140648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.140614    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-wnbx7\""
Jul 18 02:38:04.140648 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.140615    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 18 02:38:04.144548 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.144502    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"]
Jul 18 02:38:04.204734 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.204686    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj\" (UID: \"d463958e-fe23-4533-837b-5280cf38e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:04.306026 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.305987    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj\" (UID: \"d463958e-fe23-4533-837b-5280cf38e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:04.306606 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.306586    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj\" (UID: \"d463958e-fe23-4533-837b-5280cf38e2b5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:04.450633 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.450540    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:04.585693 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:04.585668    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"]
Jul 18 02:38:04.587458 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:38:04.587429    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd463958e_fe23_4533_837b_5280cf38e2b5.slice/crio-8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68 WatchSource:0}: Error finding container 8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68: Status 404 returned error can't find the container with id 8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68
Jul 18 02:38:05.567688 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:05.567593    2560 generic.go:356] "Generic (PLEG): container finished" podID="d463958e-fe23-4533-837b-5280cf38e2b5" containerID="4760d924bf6c2ffeb7654f9c186df7b2ff313c96e8aede5f3b29304bce314a4d" exitCode=0
Jul 18 02:38:05.568060 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:05.567671    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj" event={"ID":"d463958e-fe23-4533-837b-5280cf38e2b5","Type":"ContainerDied","Data":"4760d924bf6c2ffeb7654f9c186df7b2ff313c96e8aede5f3b29304bce314a4d"}
Jul 18 02:38:05.568060 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:05.567733    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj" event={"ID":"d463958e-fe23-4533-837b-5280cf38e2b5","Type":"ContainerStarted","Data":"8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68"}
Jul 18 02:38:06.710429 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:06.710400    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:06.828223 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:06.828136    2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca\") pod \"d463958e-fe23-4533-837b-5280cf38e2b5\" (UID: \"d463958e-fe23-4533-837b-5280cf38e2b5\") "
Jul 18 02:38:06.828503 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:06.828477    2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d463958e-fe23-4533-837b-5280cf38e2b5" (UID: "d463958e-fe23-4533-837b-5280cf38e2b5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 18 02:38:06.929701 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:06.929661    2560 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d463958e-fe23-4533-837b-5280cf38e2b5-maas-api-service-ca\") on node \"ip-10-0-129-141.ec2.internal\" DevicePath \"\""
Jul 18 02:38:07.579112 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:07.579078    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj" event={"ID":"d463958e-fe23-4533-837b-5280cf38e2b5","Type":"ContainerDied","Data":"8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68"}
Jul 18 02:38:07.579112 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:07.579101    2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"
Jul 18 02:38:07.579333 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:07.579128    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8562de496041af197e791e5f9e84708a3c8b05d36f5f8352eb195f5226f4ea68"
Jul 18 02:38:17.394206 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:17.389962    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:38:19.230039 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:19.229993    2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"]
Jul 18 02:38:19.233194 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:19.233170    2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-4b0cf7-j52xj"]
Jul 18 02:38:21.116850 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:21.116816    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d463958e-fe23-4533-837b-5280cf38e2b5" path="/var/lib/kubelet/pods/d463958e-fe23-4533-837b-5280cf38e2b5/volumes"
Jul 18 02:38:39.894419 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:39.894381    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-m72zp"]
Jul 18 02:38:55.324952 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:55.324919    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cddc5ddd-7zg4j_424e896d-b4b3-44ee-b24c-663b84849850/manager/0.log"
Jul 18 02:38:55.332150 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:55.332129    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cddc5ddd-7zg4j_424e896d-b4b3-44ee-b24c-663b84849850/copy-manifests/0.log"
Jul 18 02:38:58.215925 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:58.215894    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-m72zp_eef6c496-1b11-4022-9053-e972d7f94589/limitador/0.log"
Jul 18 02:38:58.947634 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:58.947601    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cb7b95cb-mcbdj_98102292-5e2d-4e91-be08-ac0d6041a6f5/kube-auth-proxy/0.log"
Jul 18 02:38:59.543102 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:38:59.543072    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79699b7d4d-l2gv2_e064ccab-2a4b-4ad3-a893-224c1cc5eca0/router/0.log"
Jul 18 02:39:00.136502 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:00.136472    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-cvn4q_297ec2cd-bf1e-4aab-8d2c-aeca58d28ff0/main/0.log"
Jul 18 02:39:00.256106 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:00.256070    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtgp99_0a9e7db5-b6cf-4905-8049-cb0ba3d21c15/main/0.log"
Jul 18 02:39:00.497319 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:00.497217    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7sr7z_0c8be9d2-f816-4afe-949e-3c7f8d760be8/main/0.log"
Jul 18 02:39:08.947605 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:08.947572    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-jcgp4_8ea73327-3d50-4b1a-b81e-8b49f3c91f73/global-pull-secret-syncer/0.log"
Jul 18 02:39:09.071311 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:09.071282    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zmbg6_d7e01843-2427-4a79-946a-d0ab277a748b/konnectivity-agent/0.log"
Jul 18 02:39:09.097403 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:09.097363    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-141.ec2.internal_b26391ef7c32d7e62df830c550d1274c/haproxy/0.log"
Jul 18 02:39:14.034765 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:14.034733    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-m72zp_eef6c496-1b11-4022-9053-e972d7f94589/limitador/0.log"
Jul 18 02:39:15.714547 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.714507    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/alertmanager/0.log"
Jul 18 02:39:15.738903 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.738876    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/config-reloader/0.log"
Jul 18 02:39:15.758706 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.758680    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/kube-rbac-proxy-web/0.log"
Jul 18 02:39:15.780431 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.780401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/kube-rbac-proxy/0.log"
Jul 18 02:39:15.803501 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.803473    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/kube-rbac-proxy-metric/0.log"
Jul 18 02:39:15.824845 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.824820    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/prom-label-proxy/0.log"
Jul 18 02:39:15.847358 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.847332    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_08f47886-ac01-4a5a-b753-82f753ac2623/init-config-reloader/0.log"
Jul 18 02:39:15.905687 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:15.905596    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-bbl66_372aca26-204b-4d70-911c-758204b8df2f/cluster-monitoring-operator/0.log"
Jul 18 02:39:16.010738 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.010711    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5bf7cb7d99-gc2cz_16550513-a742-4c93-84b8-dae085242159/metrics-server/0.log"
Jul 18 02:39:16.037771 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.037739    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-99895_42a460ee-1480-45e6-b0b5-046853c97a7a/monitoring-plugin/0.log"
Jul 18 02:39:16.219276 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.219190    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spmxg_b429c098-3403-4f95-9052-fe7a8b995170/node-exporter/0.log"
Jul 18 02:39:16.267409 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.267388    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spmxg_b429c098-3403-4f95-9052-fe7a8b995170/kube-rbac-proxy/0.log"
Jul 18 02:39:16.287573 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.287549    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-spmxg_b429c098-3403-4f95-9052-fe7a8b995170/init-textfile/0.log"
Jul 18 02:39:16.424821 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.424793    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/prometheus/0.log"
Jul 18 02:39:16.452455 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.452429    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/config-reloader/0.log"
Jul 18 02:39:16.476332 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.476241    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/thanos-sidecar/0.log"
Jul 18 02:39:16.509508 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.509480    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/kube-rbac-proxy-web/0.log"
Jul 18 02:39:16.536336 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.536309    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/kube-rbac-proxy/0.log"
Jul 18 02:39:16.562901 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.562860    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/kube-rbac-proxy-thanos/0.log"
Jul 18 02:39:16.598296 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.598275    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_43839636-be49-4df1-8ccf-5f1eec551331/init-config-reloader/0.log"
Jul 18 02:39:16.636388 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.636358    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-xkvkl_fc902078-3551-4bfe-b516-922b169b69f4/prometheus-operator/0.log"
Jul 18 02:39:16.667315 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.667284    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-xkvkl_fc902078-3551-4bfe-b516-922b169b69f4/kube-rbac-proxy/0.log"
Jul 18 02:39:16.744033 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.743962    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dd4c5b74-zv99p_0cfe2ce3-d22a-4f46-919d-955e58c93673/telemeter-client/0.log"
Jul 18 02:39:16.767132 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.767089    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dd4c5b74-zv99p_0cfe2ce3-d22a-4f46-919d-955e58c93673/reload/0.log"
Jul 18 02:39:16.790960 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:16.790931    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76dd4c5b74-zv99p_0cfe2ce3-d22a-4f46-919d-955e58c93673/kube-rbac-proxy/0.log"
Jul 18 02:39:18.272174 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:18.272142    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-xjzx2_fc38c1c6-cf13-4fe5-b413-5d01b40954d0/networking-console-plugin/0.log"
Jul 18 02:39:19.026386 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.026345    2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"]
Jul 18 02:39:19.026861 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.026841    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d463958e-fe23-4533-837b-5280cf38e2b5" containerName="revoke-keys"
Jul 18 02:39:19.026965 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.026864    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="d463958e-fe23-4533-837b-5280cf38e2b5" containerName="revoke-keys"
Jul 18 02:39:19.027022 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.026998    2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="d463958e-fe23-4533-837b-5280cf38e2b5" containerName="revoke-keys"
Jul 18 02:39:19.030145 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.030123    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.032843 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.032820    2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hsnmv\"/\"default-dockercfg-wpgd5\""
Jul 18 02:39:19.032946 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.032875    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hsnmv\"/\"kube-root-ca.crt\""
Jul 18 02:39:19.032946 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.032935    2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hsnmv\"/\"openshift-service-ca.crt\""
Jul 18 02:39:19.038179 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.038157    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"]
Jul 18 02:39:19.120897 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.120868    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-podres\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.120897 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.120901    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-lib-modules\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.121110 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.120925    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-proc\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.121110 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.120947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn55n\" (UniqueName: \"kubernetes.io/projected/da38e7c3-0d6b-48c0-b83c-373e8184dff4-kube-api-access-zn55n\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.121110 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.121071    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-sys\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.221923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.221886    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-sys\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.221975    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-podres\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222006    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-lib-modules\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222027    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-proc\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222030    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-sys\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222109 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222053    2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn55n\" (UniqueName: \"kubernetes.io/projected/da38e7c3-0d6b-48c0-b83c-373e8184dff4-kube-api-access-zn55n\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222369 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222152    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-proc\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222369 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222179    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-lib-modules\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.222369 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.222185    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/da38e7c3-0d6b-48c0-b83c-373e8184dff4-podres\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.231227 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.231203    2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn55n\" (UniqueName: \"kubernetes.io/projected/da38e7c3-0d6b-48c0-b83c-373e8184dff4-kube-api-access-zn55n\") pod \"perf-node-gather-daemonset-4tc48\" (UID: \"da38e7c3-0d6b-48c0-b83c-373e8184dff4\") " pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.341699 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.341597    2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.440257 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.440227    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9556bff77-qtjkf_bc2d391e-6881-41fd-978b-b90eb0155969/console/0.log"
Jul 18 02:39:19.476953 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.476930    2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"]
Jul 18 02:39:19.479183 ip-10-0-129-141 kubenswrapper[2560]: W0718 02:39:19.479158    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podda38e7c3_0d6b_48c0_b83c_373e8184dff4.slice/crio-31c44c717683af6a22722e180973f48110e5c77157a582c921065016ad8090ce WatchSource:0}: Error finding container 31c44c717683af6a22722e180973f48110e5c77157a582c921065016ad8090ce: Status 404 returned error can't find the container with id 31c44c717683af6a22722e180973f48110e5c77157a582c921065016ad8090ce
Jul 18 02:39:19.929559 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.929498    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48" event={"ID":"da38e7c3-0d6b-48c0-b83c-373e8184dff4","Type":"ContainerStarted","Data":"a609df333798025a2fbb299c42d99c6d2321b864bd924acefacc97264b888926"}
Jul 18 02:39:19.929795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.929586    2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48" event={"ID":"da38e7c3-0d6b-48c0-b83c-373e8184dff4","Type":"ContainerStarted","Data":"31c44c717683af6a22722e180973f48110e5c77157a582c921065016ad8090ce"}
Jul 18 02:39:19.929795 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.929613    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:19.948511 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:19.948455    2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48" podStartSLOduration=0.94843727 podStartE2EDuration="948.43727ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-18 02:39:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-18 02:39:19.946054564 +0000 UTC m=+2929.268434735" watchObservedRunningTime="2026-07-18 02:39:19.94843727 +0000 UTC m=+2929.270817434"
Jul 18 02:39:20.116039 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:20.116007    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-j476l_f9cf1546-b46c-4014-8ad9-64752a889523/volume-data-source-validator/0.log"
Jul 18 02:39:21.150505 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:21.150479    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tltqd_64a2e455-05f6-481b-98fc-20d3f9b91197/dns/0.log"
Jul 18 02:39:21.171680 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:21.171650    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tltqd_64a2e455-05f6-481b-98fc-20d3f9b91197/kube-rbac-proxy/0.log"
Jul 18 02:39:21.304173 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:21.304137    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qcblm_93de963b-7c22-4e91-a404-82baa95cafc7/dns-node-resolver/0.log"
Jul 18 02:39:21.885429 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:21.885404    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q7wjb_ac9788af-d9b4-4af3-8220-d2b97fdc56d0/node-ca/0.log"
Jul 18 02:39:22.991133 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:22.991102    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cb7b95cb-mcbdj_98102292-5e2d-4e91-be08-ac0d6041a6f5/kube-auth-proxy/0.log"
Jul 18 02:39:23.162661 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:23.162628    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79699b7d4d-l2gv2_e064ccab-2a4b-4ad3-a893-224c1cc5eca0/router/0.log"
Jul 18 02:39:23.829228 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:23.829197    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5dlq8_3bb385be-cac8-481e-a541-82de95f3f191/serve-healthcheck-canary/0.log"
Jul 18 02:39:24.493407 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:24.493385    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmp2v_4a8317e4-bba8-4b86-80b0-f1ea0208f85f/kube-rbac-proxy/0.log"
Jul 18 02:39:24.515367 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:24.515342    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmp2v_4a8317e4-bba8-4b86-80b0-f1ea0208f85f/exporter/0.log"
Jul 18 02:39:24.538755 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:24.538726    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cmp2v_4a8317e4-bba8-4b86-80b0-f1ea0208f85f/extractor/0.log"
Jul 18 02:39:25.945511 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:25.945484    2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hsnmv/perf-node-gather-daemonset-4tc48"
Jul 18 02:39:27.197926 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:27.197894    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cddc5ddd-7zg4j_424e896d-b4b3-44ee-b24c-663b84849850/manager/0.log"
Jul 18 02:39:27.217602 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:27.217574    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-75cddc5ddd-7zg4j_424e896d-b4b3-44ee-b24c-663b84849850/copy-manifests/0.log"
Jul 18 02:39:28.811803 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:28.811777    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64d778cd56-c8szz_e289052c-c4d3-4c41-9eb5-ee038b41d038/manager/0.log"
Jul 18 02:39:28.867958 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:28.867930    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-4756v_c2ca91c3-0098-47eb-847c-47b82c3be8b9/openshift-lws-operator/0.log"
Jul 18 02:39:29.523410 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:29.523320    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cb7b95cb-mcbdj_98102292-5e2d-4e91-be08-ac0d6041a6f5/kube-auth-proxy/0.log"
Jul 18 02:39:29.644507 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:29.644474    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79699b7d4d-l2gv2_e064ccab-2a4b-4ad3-a893-224c1cc5eca0/router/0.log"
Jul 18 02:39:30.955204 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:30.955109    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-65cb7b95cb-mcbdj_98102292-5e2d-4e91-be08-ac0d6041a6f5/kube-auth-proxy/0.log"
Jul 18 02:39:31.092193 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:31.092164    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79699b7d4d-l2gv2_e064ccab-2a4b-4ad3-a893-224c1cc5eca0/router/0.log"
Jul 18 02:39:34.376390 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:34.376355    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-t4m44_d8f740af-542b-40c4-be19-372fe4ed6873/kube-storage-version-migrator-operator/1.log"
Jul 18 02:39:34.378313 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:34.378286    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-t4m44_d8f740af-542b-40c4-be19-372fe4ed6873/kube-storage-version-migrator-operator/0.log"
Jul 18 02:39:35.343550 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.343506    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/kube-multus-additional-cni-plugins/0.log"
Jul 18 02:39:35.367303 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.367219    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/egress-router-binary-copy/0.log"
Jul 18 02:39:35.389394 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.389368    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/cni-plugins/0.log"
Jul 18 02:39:35.409778 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.409753    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/bond-cni-plugin/0.log"
Jul 18 02:39:35.429923 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.429900    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/routeoverride-cni/0.log"
Jul 18 02:39:35.449848 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.449817    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/whereabouts-cni-bincopy/0.log"
Jul 18 02:39:35.470869 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.470844    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gmx26_6facbd29-cddd-4b48-80bc-76682898ab95/whereabouts-cni/0.log"
Jul 18 02:39:35.855396 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.855363    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-l7sd8_a1601f09-60c2-482f-bd62-846866d5bbdf/kube-multus/0.log"
Jul 18 02:39:35.902657 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.902623    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6r66r_c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e/network-metrics-daemon/0.log"
Jul 18 02:39:35.919837 ip-10-0-129-141 kubenswrapper[2560]: I0718 02:39:35.919808    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-6r66r_c284b922-9ec6-4f4e-8a2d-b3f99ace8b2e/kube-rbac-proxy/0.log"